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1 Executive Summary 

On behalf of our partners, the Kempe Center at the University of Colorado and Joyce James Consulting, 
Chapin Hall Center for Children is pleased to submit a response to the RFP requesting an evaluation of the 
Predictive Risk Model (PRM) being developed by the Allegheny County Department of Human Services 
(DHS). The team of researchers assembled has the requisite research, evaluation, and operational 
experience, with specific experience developing predictive models, decision-making analytics, advanced 
statistical skills, and deep experience designing evaluations that target public child welfare programs. In 
addition, the team brings a wide range of public sector leadership experience to its understanding of how 
PRM supports decision-making in a child protective services context. 

The proposed evaluation covers both the impact and the process study. The impact evaluation will be led 
by Chapin Hall, with support from the Kempe Center and Joyce James Consulting. Joyce James 
Consulting will assume responsibility for the process evaluation, with guidance coming from Chapin Hall 
and the Kempe Center. Overall fiduciary and project management will come from Chapin Hall, which will 
serve as the prime contractor. 

Per our proposal, the evaluation will unfold in four phases. For the first phase, we propose in-depth 
discussions with the involved staff from DHS and PRM developers. During this phase, we propose a series 
of teleconferences in order to reach a clear understanding the PRM and the implementation strategy. In 
particular, we want to better appreciate the decision-making context, with specific focus on work flow, the 
delivery of PRM results to call screeners, the decisions DHS expects to influence with the PRM, and the 
outcomes of interest given a decision to screen in or screen out. Finalization of the evaluation plan is 
embedded in this phase of the work. A draft evaluation plan, including instrumentation and administrative 
data review, will be presented to the DHS leadership and PRM developers for review, with revisions to the 
plan introduced after the review. In general, our proposal calls for a difference-in-difference analysis that 
compares the screening decisions during pre- and post-PRM periods. Our evaluation strategy embeds 
Decision-Making Ecology and escalation/de-escalation frameworks into the overall design. The planning 
phase will conclude with an IRB review of the evaluation plan. 

Data collection will occur during the second phase. Administrative data sources will be identified and 
aligned with the evaluation plan. As a general matter this involves organizing contact with child protective 
services into event histories that organize the sequence, timing, and duration of events into temporal order 
so that time and sequence dependencies within the event histories can be identified. Alignment also 
requires operationalization of the True Positives, True Negatives, False Positives, and False Negatives with 
the administrative data. For the process/implementation study, we propose two types of surveys: 
prospective surveys that examine decision-making prior to PRM implementation plus two retrospective 
surveys designed to gather feedback during implementation. Focus groups are also described. Results will 
be coded using either NVivo or ATLAS.TI. Where necessary we will test for interrater reliability. 

Data analysis will take place during the third phase. Although the proposal calls for a summative evaluation, 
we understand that the projects such as this are intended to improve outcomes for children. To that end, 
our proposal describes a formative stage in which early results from both the quantitative and qualitative 
studies will be brought to the attention of DHS leadership at appropriate points in time so that the 
implementation plan can be revised according to what is learned during the project’s earliest phases. We 
are particularly sensitive to the fact that although the PRM is designed to improve decision-making, there is 
a possibility that the risk to children will increase (i.e., an increase in false negatives), and that there is an 
even greater likelihood that false positives will increase with a corresponding cost impact. For this reason, 
we will fast-track the quantitative analysis so that results from the PRM implementation are available just as 
soon as the first wave of reliable data are available. In the event we see a significant increase in risk or 
workload, we will offer recommendations to DHS about how best to continue with the demonstration. 

The fourth and final step involves report preparation. Assuming a twelve-month timeframe, we will submit a 
preliminary report at the end of month eleven for review by the model developers and DHS leadership. 

Upon review, we will amend the draft report and then submit the final report at the end of the project period. 
In addition, we will plan formative summaries of progress, in consultation with DHS, so that implementation 
is guided by evidence. 
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2 Impact Study Narrative 

2.1 Organizational Experience 

2.1.1 Specific experience in the study of decision-making 

2.1.1.1 Chapin Hall Center for Children at the University of Chicago 

The Chapin Hall Center for Children is one of the world’s pre-eminent research centers focused on 
vulnerable children, with a specific emphasis on children and families served by the nation’s child welfare 
system. From its inception in 1986, Chapin Hall has worked to link rigorous research with policy and 
practice for the purpose of strengthening the systems that serve children, families, and communities. In 
pursuit of that agenda, Chapin Hall maintains a diverse research portfolio that is methodologically varied 
and theoretically grounded. In addition to the Chapin Hall’s core research program, the Center supports 
knowledge translation. Of particular interest, Chapin Hall’s Center for State Child Welfare Data supports 
training and technical assistance in advanced analytics, continuous quality improvement, and evaluation 
design. Importantly, Chapin Hall is affiliated with the University of Chicago. The affiliation gives Chapin Hall 
staff access to leading methodologists and statisticians and its extensive library resources. Finally, staff of 
the Center for State Child Welfare Data have extensive experience working with state administrative data, 
including Allegheny’s data warehouse. Our experience includes the development and maintenance of 
integrated data systems similar to the one used to support the Predictive Risk Model. 

Staff at Chapin Hall and the Center for State Child Welfare data have a long history of designing 
evaluations of public child welfare programs for federal, state and local government agencies. The work 
has been both basic and applied in nature. The Big Data revolution in child welfare was started at Chapin 
Hall in the early 1980s when the first longitudinal event history files were developed from Illinois child and 
family tracking system. With those data, staff at Chapin Hall went on to pioneer the use of event history 
analysis for purposes of understanding what happens to children in contact with the child welfare system. 
Early studies included multistate comparisons of child outcomes (e.g., length of stay, reentry, placement 
stability), the role of local ecology vis a vis the incidence of child abuse and neglect and the demand for 
foster care. Much of that work has had a direct impact on the outcomes measures adopted by the U. S. 
Department of Health and Human Services as part of the Child and Family Service Reviews. The Center’s 
basic research carries over into applied settings, with an emphasis on both summative and formative 
evaluations of service innovations. Chapin Hall researchers have designed experimental, quasi- 
experimental, and observational evaluation studies of innovative child welfare programs including family 
preservation, the implementation of evidence-based interventions, and decision-making using family/group 
decision models. 

Chapin Hall’s diverse experience is particularly well suited for the PRM evaluation. First and foremost, we 
have extensive experience with the DHS warehouse data having used that data to build longitudinal event- 
based files for the evaluation of the county’s system of care, wraparound, and family support centers. The 
principles behind those designs are particularly relevant in the case of the PRM. Predictive risk modeling 
relies heavily on what happens next in the case trajectory of a child who has been screened in (or not) 
inasmuch as false positives and negatives are determined on the basis of subsequent events. Chapin Hall 
staff are already in possession of the computer algorithms needed to organize the analytical files. With 
those files and other similar files, Chapin Hall researchers have expanded the use of multi-level, discrete 
time models that are especially well-suited to understanding how decision processes and their 
consequences are affected by the characteristics of children, caregivers, providers, and contextual factors. 
Finally, Chapin Hall has considerable experience carrying out and supervising research that examines the 
implementation of programs and other innovations that target decision-making processes. Working in 
settings as diverse as Tennessee, New York City, and Michigan, our portfolio includes research that links 
data collected qualitatively with administrative data at the worker, child/family, and agency level so as to 
understand how workers use time and whether time use affects decision making and outcomes. With 
respect to Allegheny’s IV-Waiver project, Chapin Hall is leading the fiscal analysis, consulting on the 
outcome analysis, and implementing the analytical files. 

2.1.1.2 Kempe Center for the Prevention and Treatment of Child Abuse and Neglect 

For more than 40 years, the Kempe Center has promoted understanding, knowledge, and best practices to 
prevent and treat child abuse and neglect locally, nationally, and internationally. Kempe fulfills its mission 
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through research and evaluation; training, technical assistance, and education; and service delivery, 
advocacy, and consultation. Kempe is administratively located within the Department of Pediatrics at the 
University of Colorado Anschutz Medical Campus in Aurora, Colorado. Kempe’s 100 faculty and staff are 
experienced in designing, implementing, evaluating, and disseminating a wide range of evidence-based 
programs and initiatives (e.g., differential response, trauma-focused mental health treatment, family group 
decision making, risk and safety assessment, and community-based child abuse prevention) to prevent and 
treat child abuse and neglect as well as improve the health and well-being of children and families. When 
several staff members from the American Humane Association joined the faculty of Kempe in 2012, their 
addition made Kempe distinctive among similar organizations in the United States and internationally by 
increasing its breadth of expertise related to child welfare practice, policy, systems, and research. 

With respect to the work proposed herein, Kempe faculty are known for their groundbreaking research 
using an ecological approach to understanding factors affecting child welfare staff decision making and 
outcomes. This experience includes 1) the identification of case factors that influence the likelihood that 
child maltreatment will recur or that families and/or children will be the subject of repeat referral or re¬ 
reports; 2) the identification of agency, worker, and case characteristics’ that influence decision making 
practices, 3) the impact of policies, practices, and demographics on states’ child maltreatment recurrence 
rates, and 4) using research findings to develop safety and risk assessment tools, workforce development, 
and policies. These efforts have yielded tools and models that predict variations in outcomes from the 
individual to the aggregate level such as counties or states. The experience entailed will be leveraged to 
inform the design and implementation of the evaluation in ways that are both highly specific to the 
evaluation needs of the Allegheny PRM evaluation, but also will reflect lessons learned from a range of 
prior evaluations of public child protection systems. 

Kempe staff are highly accustomed to working with the myriad professionals across the multiple systems 
that work to protect children and support vulnerable families. Kempe staff have conducted focus groups, 
interviews, and surveys with a spectrum of public and private child welfare staff (administrators, program 
directors, supervisors, caseworkers, information technology staff and analysts, administrative staff, and 
volunteers) at federal, state, and county levels. Given the interconnection between the formal child welfare 
system and other human service systems, the Kempe team has also acquired significant experience 
gathering information from and involving other stakeholders, including parents/caregivers and 
representatives from mental health, substance abuse, prevention, juvenile justice, courts, and 
developmental disabilities. Kempe staff will draw on these experiences as they collaborate with Chapin Hall 
and Joyce James Consulting team members to evaluate the decision support and predictive analytic tools. 
The expertise and accomplishments of Kempe’s faculty reflect a deep commitment to conducting research 
and translating it into practice. Moreover, the development and application of these and other analyses 
have involved the specification of data extracts from child welfare, mental health, heath, benefits and other 
information systems in many states, construction of event data, and the use of a range of analytic 
procedures. Kempe staff are well-versed in a variety of methodological approaches and have led studies 
involving randomized control trials, propensity-score comparisons and other quasi-experimental designs. 
Other specific program and system evaluations conducted by Kempe staff include the impact of differential 
response, family group decision making, and factors influencing child welfare disparities. The depth and 
breadth of their experiences reflects a firm commitment to the use of evaluation to inform policy and 
practice decisions, advance the understanding of workforce development, develop models of agency 
finance and costs, improve outcomes aimed at strengthening child welfare agencies. 

2.1.1.3 Joyce James Consulting, Inc. (JJC) 

Joyce James Consulting (JJC), a minority woman owned firm, has had a number of years’ experience in 
conducting process evaluations that have included key informant interviews, focus groups and training in a 
number of states that include New York, Pennsylvania, Texas, Indiana and Oklahoma. In this context she 
has worked with a number of cross-system agencies. Joyce James, the owner and principal in JJC, is a 
nationally recognized expert in racial equity, social injustices, and system transformations all within a 
backdrop of decision-making. With a professional career spanning more than three decades, Ms. James 
provides consulting services to organizations and institutions at various levels in both the public and private 
sectors. In addition; as reflected on the enclosed resume, Ms. James has had many successful 
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experiences in supporting the strategic goals and objectives of child welfare systems, local education 
agencies, colleges, and universities. 

Ms. James most recently served as the Associate Deputy Executive Commissioner for the Center for 
Elimination of Disproportionality and Disparities (Center) at the Texas Health and Human Services 
Commission (HHSC). The Center was created by former HHSC Executive Commissioner Thomas Suehs. 
Ms. James was appointed to the leadership role within the Center and HHSC in September 2010 in 
recognition of Ms. James' record of strong and effective leadership and decision-making throughout her 
career and a desire to expand the Texas Model for Addressing Disproportionality and Disparities— 
developed under her leadership—to all Health and Human Services (HHS) agencies and programs. She 
was principle investigator on a number of national and local grants which had both impact and process 
evaluation components. These included funding from the Children’s Justice Act through the Texas 
Supreme Court's Permanent Commission on Children Youth and Families where pre-post surveys yielded 
both quantitative and qualitative evaluation data regarding the impact of training on the decision-making of 
school personnel as reporters of child abuse and neglect, eight grants from the Federal Office of Minority 
Health through the Center's role as State Officer of Minority Health, and funding through Texas’ state 
Medicaid office to study the immunization process for the poor where focus groups and interviews were 
conducted and surveys were administered. 

Prior to becoming the Associate Deputy Executive Commissioner at HHSC, Ms. James was the Deputy 
Commissioner for the Texas Department of Family & Protective Services (DFPS). At DFPS, Ms. James 
provided leadership for the Center for Learning and Organizational Effectiveness with responsibility for 
developing and implementing training for approximately 11,000 DFPS from entry to executive level. The 
four years before that, Ms. James was the head administrator and decision-maker for Child Protective 
Services in Texas where she championed impact and process evaluations that helped solidify the view of 
caseworkers as decision-makers. She published a number of articles with others on the Decision-making 
Ecology that resulted from these efforts. Three interventions that were funded by the legislature were 
evaluated from and impact and process perspective. They were Family Group Decision-Making, The 
Relative Care Assistance Program and the Strengthening Families Initiative. A fourth, funded through 
Casey Family Programs and the American Humane Association, used a Decision-Making Ecology 
approach, such as that suggested here. The study used a multi-level model based on data from surveys 
(process) linked to administrative data containing actual worker decisions. 

2.1.2 Information about personnel 
2.1.2.1 Chapin Hall Center for Children 

Fred Wulczyn, Ph.D., is a senior research fellow at Chapin Hall and founding director of the Center for 
State Child Welfare Data. Trained at the University of Chicago, Dr. Wulczyn has focused his work on the 
use of electronic/computerized case records for purposes of research and evaluation. He designed the 
original longitudinal foster care placement files, now in use at Chapin Hall since 1980. In the process of 
continually updating the designs he developed, Dr. Wulczyn has integrated maltreatment, public 
assistance, Medicaid, birth records, education, and mental records with placement data to provide a 
comprehensive view of service involvement. In addition to his work designing databases, Dr. Wulczyn has 
extensive experience analyzing those data. Geographic information, hierarchical modeling, event history 
models, and simulation fall within the range of his skills. Having worked as a senior government official in 
New York State, and later as a consultant to the U.S. Department of Health and Human Services during the 
Obama Administration, Dr. Wulczyn brings a deep appreciation for decision-making under conditions of 
uncertainty. 

Lijun Chen, Ph.D., is a senior researcher at Chapin Hall. A sociologist by training, Dr. Chen has for the 
past 12 years worked extensively with state administrative data, developing model statistical approaches to 
the problem of understanding child level outcomes. He specific areas of expertise include hierarchical 
models, event history analysis, and structural equations. Dr. Chen has experience with a wide range of 
statistical programs including SAS, SPSS, State, and R. He also works closely with others at Chapin Hall 
leading our record linkage projects. In that regard, he has linked birth records to CPS data and placement 
data. 
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Jianyu Wang, Ph.D., is the data base administrator for the Center for State Child Welfare Data. In that role 
he oversees data management, which includes data receipt, data processing, and backup. He also 
maintains the Data Center’s online web portal. He too has experience creating longitudinal data records 
from state SACWIS and SACWIS-like systems. He performed much of the file manipulation with the 
Allegheny county data when it first came to Chapin Hall. He also just completed a project in Delaware that 
called for creating an event history file from CPS and placement data. 

2.1.2.2 Kempe Center for the Prevention and Treatment of Child Abuse and Neglect 

John Fluke, PhD, is Associate Director of System Research and Evaluation at the Kempe Center for the 
Prevention and Treatment of Child Abuse and Neglect and Associate Professor at the University of 
Colorado School of Medicine. With a career focused on the areas of child welfare and mental health 
services for children, he is internationally recognized as a researcher specializing in assessing and 
analyzing decision making in human services delivery systems. His contributions to Decision Making 
Ecology are widely recognized. He is also active in the area of national child maltreatment data collection 
systems and analysis and has worked with data collection programs in the Balkans, Canada, Saudi Arabia, 
the US, and for UNICEF. He is also known for his innovative and informative research and evaluation work 
in the areas of child maltreatment prevalence, child welfare administrative data analysis, workload and 
costing, and performance and outcome measurement for children and family services. 

Dana Hollinshead, PhD, is an Assistant Research Professor at the Kempe Center for the Prevention and 
Treatment of Child Abuse and Neglect, University of Colorado School of Medicine, Denver, CO. Dr. 
Hollinshead is a graduate of the Heller School of Social Policy and Public Management at Brandeis 
University. She has extensive child welfare project management, research, program evaluation, and policy 
analysis experience utilizing advanced quantitative and qualitative data collection, analysis. Owing to her 
experience with in working with child welfare agencies in states and having worked with the children’s 
bureau Dr. Hollinshead has gained insight into federal, state, and local government perspectives on child 
welfare programs, policies, practices, administrative data, performance measurement, program 
improvement, and system reform efforts. Her expertise extends to examining a comprehensive array of 
factors affecting child welfare outcomes including decision-making. Dr. Hollinshead is also highly 
experienced in disseminating research and evaluation results and has made many presentations, 
conducted trainings, provided technical assistance, and prepared many reports and publications. 

2.1.2.3 Joyce James Consulting 

Joyce James, LMSW, the owner and principal in JJC, is a nationally recognized expert in racial equity, 
social injustices, and system transformations all within a backdrop of decision-making. With a professional 
career spanning more than three decades, Ms. James provides consulting services to organizations and 
institutions at various levels in both the public and private sectors. In addition; as reflected on the enclosed 
resume, Ms. James has had many successful experiences in supporting the strategic goals and objectives 
of child welfare systems, local education agencies, colleges, and universities. Ms. James most recently 
served as the Associate Deputy Executive Commissioner for the Center for Elimination of Disproportionality 
and Disparities (Center) at the Texas Health and Human Services Commission (HHSC). Prior to becoming 
the Associate Deputy Executive Commissioner at HHSC, Ms. James was the Deputy Commissioner for the 
Texas Department of Family & Protective Services (DFPS). At DFPS, Ms. James provided leadership for 
the Center for Learning and Organizational Effectiveness with responsibility for developing and 
implementing training for approximately 11,000 DFPS from entry to executive level. The four years before 
that, Ms. James was the head administrator and decision-maker for Child Protective Services in Texas 
where she championed impact and process evaluations that helped solidify the view of caseworkers as 
decision-makers. She published a number of articles with others on the Decision-making Ecology that 
resulted from these efforts. 

Donald Baumann, Ph.D. has over thirty years’ experience designing and managing large-scale research 
projects as both a Principal Investigator and as a Project Director. He has been on the faculties of the 
University of Texas, Trinity University and Saint Edwards University. While at the University of Texas he 
began his exploration of decision-making with consequences while working with the police and Travis 
County Prosecutors Office. He has directed numerous national multi-year research and evaluation projects 
over the years, most of which have been focused on understanding decision-making or finding solutions to 
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augment that process (e.g., risk assessment). He is retired from the Texas Department of Family and 
Protective Services where he was head of the Evaluation Section of Child Protective Services. While there 
for twenty years he helped develop The Decision-Making Ecology which has been empirically applied to 
caseworker burnout and performance, the intake process, the substantiation decision, the decision to place 
children into care and the decision to reunify children with their parents. The model looks at the context of 
decision-making as a process, in relation true positives and negatives as well as false positives and 
negatives as outcomes of these decisions. He currently teaches at St. Edwards University and is a 
Research Associate with Joyce James Consulting, LCC. He has written over eighty reports, articles and 
book chapters. 

Windy Hill, BA, has a 20-year career in executive leadership and management has included successful 
experience in public administration, nonprofit management, community organizing and development and 
consulting at the grassroots, state and national levels. Windy is currently an Associate Racial Equity 
Consultant with Joyce James Consulting, LLC; where she provides qualitative data acquisition and 
analysis, consultation, professional development training and community organizing through processes that 
create momentum and direction for improving racial equity within systems and institutions. Windy served 
most recently as a senior policy and program specialist in the Texas Health and Human Services 
Commission (HHSC) Center for Elimination of Disproportionality and Disparities (CEDD) where she 
performed complex activities and administrative functions that supported the HHSC Associate Deputy 
Executive Commissioner’s management and oversight of the CEDD. In this role, she spearheaded a 
number of decision-making projects that involved process evaluations relying on interview and focus group 
data as well as survey data. 

2.2 Approach to Impact Evaluation 

2.2.1 Background and key research questions 

Predictive analytics refers to the use of health and related information from the past to determine the level 
of care needed in the future. In the case of Allegheny’s Predictive Risk Model, the goal is to use information 
from the past, sourced from the agency’s data warehouse, to more accurately sort reports into those that 
need further investigation and those that do not. 1 Upon investigation, if the report establishes a need for 
services, the family’s case can be accepted for services, the range of which may include placement. 
Although call screeners have access to electronic records, the PRM is designed to systematize access to 
that data. Moreover, the PRM links that information in a way that should reduce the likelihood that 1) calls 
not needing further investigation will be accepted (i.e., false positives) and 2) calls needing follow-up will be 
rejected from further review (i.e., false negatives). 

In this context, an important question for the impact evaluation is whether the actual decisions taken 1) 
improve the safety of children, and 2) reduce the rate of unnecessary responses and related agency costs. 
From the RFP we assume the PRM model(s) that are the subject of the evaluation have an established 
degree of validity based on retrospective analysis; the evaluation will focus on whether in practice the 
model does indeed improve the overall accuracy of the decisions made. 

With that background in mind, we have identified these key research questions: 

1. Is the PRM performing as intended? If the PRM is working as intended true positive and true 
negative rates will rise thereby improving the accuracy of the decisions made by screeners. 

2. Does the model’s performance differ depending on characteristics of the child, the family, or 
contextual factors? 

3. Does the model’s performance depend on the screener? To the extent model performance varies, 
are there attributes of the screeners associated with that variance. 


1 Throughout our proposal we focus our discussion on the screen in/screen out decision. We understand there are other 
decisions involved in the process and those decisions may, directly or indirectly, affect whether the decision to screen in or out is 
later adjusted in light of what happens with a particular case. We also know that the Allegheny County uses the GPS/CPS 
distinction and that the latitude afforded to the county with regard to the decisions within the county’s discretion may be affected 
by the GPS/CPS distinction. We further assume that these details will be worked out with the DHS staff and the PRM team. 
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4. Given the cost of providing services to cases later identified as false positives and the costs 
avoided by screening out cases, does the PRM lead to a reduction in overall costs when compared 
to business as usual costs. 

2.2.2 Theoretical Framework 

Decision-Making Ecology. As described in the RFP, the PRM is designed to more accurately predict 
risk. As such the PRM becomes one factor amongst many that influence the decisions decision-makers 
make. The DME (Baumann, Dalgleish, Fluke, & Kern, 2011) is a framework for organizing child welfare 
decision-making research and evaluation, and places the topic squarely in the context of actual child 
welfare protective service operations. Decisions like intake take place within an agency culture; in other 
words, there is a systemic context that regulates the case decisions made by the management and staff of 
the agency. The systemic context for decision-making includes a set of decision-making influences such 
case characteristics as well as external, organizational, and caseworker factors that combine in various 
ways to influence decisions and outcomes. Further, these factors are constantly operating to shift decision 
thresholds over time as a result of feedback consistent with systems theory. The specific factors found to 
influence agency decision thresholds and the willingness to take action represent leverage points that can 
be translated into policy, workforce development, and practice including the implementation of PRM. 
Escalation/De-escalation. In order to understand whether decision making improves (i.e, becomes 
more accurate) when a PRM is being used, a classification scheme that differentiates the decisions made 
has to be adopted. The escalation/de-escalation provides such a framework (Mansell, Ota, Erasmus, & 
Marks, 2011). Described in greater detail below, the framework provides a way of describing decision 
outcomes. Specifically, once a report of concern is received, the action options are to escalate 
(investigate/assess) or de-escalate (not investigate/assess). The two possible outcomes for escalation are 
to make a finding of maltreatment, or to determine that no maltreatment took place. For de-escalation the 
outcomes are no subsequent findings of maltreatment, or alternatively that subsequent maltreatment is 
found (re-reporting with substantiation (Fluke et al. 2008)). Although operationalizing escalation and de- 
escalation conditions and their outcomes are clearly dependent on agency specific contexts, the concept 
can be adapted to evaluate the statistical validity of predictive model used in Allegheny. 

2.2.3 Analytical Strategy 

The impact study will be built around two basic analytical strategies. Described in succession below, the 
strategies are: Area Under the Curve (AUC) and discrete time random effects model (DTM). 

2.2.3.1 Area Under the Curve 

The validity of any forecasting approach for decision-making is typically addressed through attention to the 
concept of errors in the prediction or classification of events. These errors are often expressed in their 
simplest form as a fourfold contingency table (sometimes referred to as a Confusion Matrix; see Figure 1): 

Figure 1: Contingency Table Describing Decision Outcomes 

True Outcome 


Predicted 

Outcome 



Maltreatment Occurred 

Maltreatment Did Not Occur 


Maltreatment 

True Positive - TP 

False Positive - FP 

Predicted Positives 
(TP +FP) 

No Maltreatment 

False Negative - FN 

True Negative-TN 

Predicted Negatives 
(FN+TN) 


Positives 

P 

Negatives 

N 

Decisions Made 
(P+N) 


Statistically speaking, in developing a predictive risk model (PRM), a key goal is to minimize the false 
positive and false negative errors, and to improve derived rate measures such as precision (TP/(TP+FP)), 
sensitivity (TP)/(TP+FN), and specificity (TN/(TN+FP). 2 There are several evaluative tests such as Area 
Under the Receiver Operator Curve (AUC) that describes the overall performance probability and estimated 
error of a PRM by determining the combined effect of sensitivity and specificity (specificity, or the rate of 


2 Precision is sometimes referred to as positive predictive value. 
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false positives). Values of the AUC are considered best as they approach 1 indicating the model is perfectly 
predicting occurrence, whereas values that approach 0.5 indicate the PRM performs no better than chance. 
The Receiver Operator Curve (ROC) from which the AUC is derived is also used to evaluate the impact of 
setting action thresholds based on the PRM modeling results. For example, using a sensitivity threshold at 
90% (referrals responded to with values from the PRM corresponding to this threshold, would theoretically 
detect 90% of the true positives) is balanced by the rate of false positives, which in turn is determined by 
the value of the ROC at the 90% threshold of sensitivity. The larger the AUC, the better the PRM is able to 
distinguish true positives from true negatives at any threshold. 

Regardless of the magnitude of the AUC, for any given PRM, there is a good chance that the observed 
results of an implemented model will be different compared to the retrospective analysis on which the PRM 
is based. An important aspect of the quantitative evaluation will be to determine the size of this difference 
through the application of ROC using the escalation/de-escalation framework during implementation. This 
will help serve to evaluate how well the PRM works during “live” intake decision-making and provide a basis 
for comparing implementation decision making under differing conditions. The AUC can also be used in a 
pre-test/post-test assessment of the PRM (see Mansell, et al. 2011). 

2.2.3.2 Discrete Time Random Effects Model 

Predictive risk models seek to improve risk classification. Screeners are making decisions based on criteria 
established in policy as well as their own assessment of risk. If the assessed risk is high, the screener 
forwards the referral with the expectation that appropriate action will follow (escalation). Similarly, low risk 
referrals are not forwarded, with the expectation that no new report will be filed (de-escalation). When 
introduced to decision-making processes. Predictive risk models are designed to reduce classification error 
and the attendant consequences. Compared to business as usual, the PRM regularizes how risk factors 
are introduced into the decision-making process. 

From the time the screen in/out decision is made (as recorded in the data warehouse), our approach will be 
to initialize a follow-up period (the start date) during which time the consequences of the decision will come 
to be known. The events of interest, as defined through the PRM modeling process, are events that reverse 
the predicted outcome. For cases screened out, subsequent reports (or reports with substantiation 
depending on DFIS/PRM definitions), represent false negatives (i.e., a reversal). For children screened in, 
the event of interest is an investigation that yields no evidence of maltreatment (as defined by the PRM and 
DHS definitions); and considered false positives (i.e., a reversal). 

In both cases, an event that reclassifies the prediction is treated as a terminal event, in much the same way 
exit from foster care marks the end of a placement spell. The corresponding date represents the stop date; 
the time between the stop and start date represents duration. Clearly, some cases will not have terminal 
events during the observation window. From the start date and through the end of the observation window 
or an event of interest, which ever comes first, we will partition time between the start and stop dates into 
discrete person period intervals. To establish the person periods, duration is divided by the interval length, 
which yields N intervals plus 1. Each interval is coded with an indicator variable denoting whether the event 
of interest was observed. For all intervals but the last the interval is coded with a zero (no event or 
censored) or a one if the event was observed. Person periods are also coded with an indicator variable 
denoting the case-specific prediction and the nature of the reversal. Using the DTM we expect to analyze 
actual intake actions as well as PRM predictions available during implementation. 

The underlying analysis in the DTM framework is nearly identical to a traditional logistic regression model. 
Importantly, the DTM framework readily accommodates predictors such as child characteristics, family 
characteristics, screener characteristics and other attributes of interest. Time is incorporated in the basic 
model and is used to determine whether events of interest are time dependent - i.e., more/less likely to 
occur as time passes. The DTM framework fits with random effects models as well. Random effects models 
are useful when the data have a nested structure as is the case when the children being screened come 
from the same neighborhood or are served by same office, as is likely the case. 

To determine whether the PRM improves decision-making, we propose using the DTM to develop DME 
adjusted models of reversals. For the sake of convention, we will refer to cases screened with the PRM 
during implementation as the treatment group and all other cases as the comparison or control group. For 
controls, pre-implementation or historical cases will be selected and aligned so as to match the same begin 



and end dates of the PRM trial (e.g., March to December). The impact evaluation will proceed in three 
phases. For phase one, we will assess the false positive/false negative rates in the control group using the 
DTM. These will serve as an unadjusted, baseline probability of a reversal. In addition, we will examine how 
reversals accumulate with the passage of time. This is particularly important in the context of the PRM trial. 
If most reversals tend to happen quickly (e.g., within a couple of months), information about the PRM’s 
utility will be available rather quickly. If reversals take time, particularly in the case of false negatives, timely 
feedback on the model’s performance will be more difficult to establish in the time set aside for the 
evaluation. 3 

We will follow this with a second phase that applies the PRM predictions to the control sample in order to 
evaluate how the screening decision would have differed had the PRM been available and followed at the 
time. A comparison of results from phases 1 baselines will provide us with an indication of baseline model 
performance. The results will also provide some indication of the extent to which various DME factors 
produce results that differ from PRM predictions. For example, in considering case factors, Allegheny 
currently requires expedited follow up with very young children. The distinction accorded young children in 
the current environment may have socialized the work force in such away that some workers are inherently 
predisposed to screening those cases in regardless of model results. For the third phase, we will combine 
both the comparison and treatment groups into a single sample, with indicator variables that identify unique 
cohorts and whether they belong to the treatment group. Again, we will assess the unconditional probability 
of reversal. Following that analysis, we will test the effect of using the PRM. For populations of otherwise 
similar children, the treatment group should have proportionately fewer reversals. In statistical terms, 
treatment group membership is coded as a dummy variable; when added to the unconditional model, the 
coefficient for the treatment group membership should be negative indicating a lower probability of reversal. 
Because the true rate of false positives is difficult to specify, we will adjust the censoring date to determine 
whether false negative rates are sensitive to the length of the follow-up period. 

Phase three will continue with analyses that examine subgroups in order to assess the extent to which 
reversal rates are linked to characteristics of cases or other DME factors. This analysis fulfills the goal of 
understanding how the decision-making ecology affects the likelihood of reversal. 

Finally, we also understand that DHS has yet to finalize its PRM implementation strategy. Because it is a 
flexible model, the DTM accommodates a wide range of implementation models. In other words, we have 
selected an evaluation strategy that will not constrain how DHS selects its implementation model. 

2.2.4 Fiscal Impact 

If using a PRM leads ultimately to more accurate decisions, there may be important fiscal consequences. 
Fewer false positives mean that fewer DHS resources, principally workers time, are used pursuing 
investigations the lead to no further involvement. More true positives mean that young people and their 
families engage services earlier with attendant benefits. Finally, false negatives can lead to delays in 
service receipt, which may increase a child’s exposure to risk, thereby complicating what likely is an 
already complicated clinical profile. 

To understand whether the PRM generates fiscal benefits, we propose a simple before and after cost 
comparison. Using the Contingency Table (described in Figure 1), we would first describe the average cost 
of providing the basic service package to the children (and families) located in each quadrant of 
Contingency Table (false positive, false negatives, etc.). The number of children in each quadrant would 
then serve as the multiplier used to determine gross cost/quadrant. The sum over each quadrant would 
generate the business as usual (BAU) cost of operation. The BAU cost would then be compared with PRM 
cost. The PRM cost would be calculated in the same way. In so far as the PRM is designed to improve 
specificity and sensitivity, the number of children in each cell should vary from BAU thereby producing a 
shift in spending pattern. The BAU minus PRM costs can be subjected to a statistical test to determine 
whether the differences are meaningful. Moreover, the Contingency Table can be used to determine 
whether cost benefits are associated with a particular cell or cells in the table. 


3 We want to acknowledge here that the screen in decisions and feedback from the investigation will likely happen in a relatively 
short timeframe, given the time to investigation completion used by Allegheny. Decisions to screen out will likely take longer to 
reverse as re-reporting may take time. We expect to use the analysis developed in these two stages to put a timeframe around 
which we can expect to observe most if not afl false negatives. 
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2.2.5 Report 

We are keenly aware of the fact the PRM models, as applied to child protection decisions, are largely 
untested. Because the affected decisions are profoundly important, a formative evaluation meant to guide 
implementation is unusually important. For this reason, we will work with DHS and the model team, from 
the outset, for the purpose of devising a reporting schedule that provides timely feedback to DHS. The main 
intent is to provide guidance as to whether false negative rates are rising. Other changes, including 
feedback from staff (based on the process study) that affect implementation fidelity are as important. We 
think routine integrated reporting from both the impact and process is key to a successful PRM trial. 

With regard to a final report, we propose delivering a draft report four weeks prior to the end of the PRM 
trial. The final report would then be delivered two weeks after receipt of comments from DHS and the PRM 
team. 

2.3 Approach to Project Management (1 page) 

2.3.1 Team Collaboration 

The partnership formed for the PRM evaluation is based on longstanding, successful working relationships. 
Chapin Hall and the Kempe Center are partners on evaluations involving the Tennessee IV-E waiver 
program and the Texas foster care redesign. In both cases, as proposed for the PRM evaluation, Chapin 
Hall serves as the prime with a subcontract to the Kempe Center. In addition, Dr. Fluke from Kempe has 
worked closely with staff from Joyce James Consulting (JJC), having published articles with Dr. Baumann 
covering a wide range of issues connected to decision-making. During her tenure with the Texas 
Department of Family and Protective Services, Joyce James worked with Dr. Fluke on issues related to 
racial disparities in the child welfare system. 

The approach to collaboration is based on clear delineation of responsibility for work products. Chapin Hall 
will assume overall management for fiduciary and substantive matters. With regard to work products, 

Chapin Hall will take lead responsibility for data extracts from the DHS warehouse. Those responsibilities 
include data acquisition, management, and security. Development of analytical files will also fall to Chapin 
Hall, but in conjunction with Allegheny staff, the PRM research team, and other members of the evaluation 
team. The design focus is on operationalization of key independent and dependent variables relative to 
what success means (e.g., the sensitivity and specificity of the PRM tool). These data will form the basis of 
the impact evaluation. All partners will be involved in data analysis and interpretation: Chapin Hall will take 
the lead with regard to selecting appropriate statistical models; Kempe and JJC will work with Chapin Hall 
to develop the conceptual framework, with a specific focus on the decision-making ecology. Interim and 
final work products will be developed jointly. 

2.3.2 Research Team 

With regard to impact, work with the research team will focus on model performance. One aim of this work 
is understanding how the model was constructed and any preliminary results so that the model structure 
can be incorporated into the impact evaluation. We will also want to know whether stratification was used. 
All members of the evaluation team will participate in this discussion. We expect initial discussions with the 
research team to take place within two weeks of award and again after an initial specification of the impact 
analysis. Thereafter, bimonthly meetings will be scheduled so that progress and preliminary results can be 
shared. Review of the draft report will occur in month 11. In the event our preliminary analysis warrants 
more frequent updates, we are prepared to support additional meetings with the research team. 

2.3.3 Allegheny Team 

Our collaboration with Allegheny team follows the model established for the research team. We see 
meetings with the research team and the Allegheny team as joint meetings so that all members of the 
collaboration are intimately involved in the decision-making. As implementation of the PRM is in the hands 
the Allegheny team, we expect to organize the meetings but co-facilitate the meetings with the Allegheny 
team. Meeting agendas will be developed jointly will all partners (Evaluation Team, Research Team and the 
Allegheny Team. 

With regard to implementation of the PRM, we understand that the PRM is designed to improve decision¬ 
making relative what the decisions would have been in the absence of decision-support technology. We 
also understand that because the PRM addresses predictions having to do with child safety, the accuracy, 


to 



specificity, sensitivity, and precision of the PRM are particularly important. For that reason, in conjunction 
with the research team, we propose working with the Allegheny Team to establish a threshold below which 
there is sufficient evidence to suggest the model is not working as intended. In the event the threshold is 
reached we will work with DHS to convene all stakeholders to undertake a formative review of the model 
with the aim of determining whether the model should be pulled from the operational context. As part of the 
decision-making process, we will work with the Allegheny team to develop an interim report schedule so as 
to inform the process. 

2.4 Requirements/limitations/challenges 

The principal requirements have to do with access to administrative data (e.g., DHS warehouse), the PRM 
model and model results, and key DHS staff and the PRM model team. Our considerable experience 
managing contracts and working directly with public agencies will minimize this challenge. 

The evaluation period will take place over a relatively short period of time that will in all likelihood be too 
short to fully observe the false negatives. With the historical data we can measure the time to reversal and 
assess how the evaluation timeframe affects the results. 

Studies that have tested the DME suggests that context matters. We are acutely aware of the fact that 
circumstances beyond the control of anyone connected with the PRM trial could substantially alter the form 
of the trial. Again, with the historical data we can judge the likely impact of unpredicted events external to 
the project. 

We expect the effort needed to secure IRB approval will be significant. Our team, with former IRB 
members, has the experience to secure the necessary IRB approvals. That said, among the challenges, 
the time needed to secure IRB approval could affect the start of the project and the evaluation. 

Last, screeners, their supervisors, and possibly investigation/assessment staff have to be willing to 
participate in the demonstration project and the evaluation. Workers may want indemnification and while 
not the responsibility of the evaluators, this may affect the implementation timeline. 

2.5 Validity/Strength of Results 

2.5.1 Escalation/De-Escalation (criterion validity through understanding outcomes) 

We anticipate using the escalation/de-escalation framework to monitor decision outcomes as described in 
the fourfold table above. We assume these outcomes will have been defined already in the PRM modeling 
process, so our intention is to seek clarity and then utilize these same outcomes to implement a difference 
within difference approach. We would want to obtain a baseline performance prior to implementation, and 
then observe changes over time. Ideally we would monitor performance monthly and test for differences in 
outcome performance using ROC methodology. We would also recommend that DHS make these data 
available operationally as feedback for intake staff and broken down to the individual or unit level in order to 
help address the variability concerns. 

2.5.2 DME (addressing reliability through understanding variability) 

The DME provides the basis for understanding sources of variability in decision-making. Typically, these 
analyses are performed at the worker or supervisory level to obtain estimates of the probability that staff or 
work units are likely to take a decision action (e.g., refer a case for a response) and are a proxy for the 
decision thresholds. These probabilities are in turn be associated with characteristics of the individual or 
group members, organizational variables and external factors (values of management, perceptions of 
liability, etc.) and risk adjusted for the case characteristics. These analyses are designed to help determine 
which sets of factors that are not case related exert the most influence on decision making. 

Over time during implementation, we would anticipate that successful identification and modification of 
leverage points aimed at reducing variability would result in changes to the statistical relationships between 
factors in the DME and decision thresholds. For example, if workload were determined to have a strong 
relationship to variability and workload was modified; then we would expect that workload would no longer 
contribute to variability. The underlying idea is to reduce variability and thus improve the reliability of actual 
decision-making behavior in the system. 
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3 Process Study Narrative 

3.1 Organizational Experience 

3.1.1 Specific experience in the study of decision-making 

3.1.1.1 Chapin Hall Center for Children at the University of Chicago 

The Chapin Hall Center for Children is one of the world’s pre-eminent research centers focused vulnerable 
children, with a specific emphasis on children and families served by the nation's child welfare system. 

From its inception in 1986, Chapin Hall has worked to link rigorous research with policy and practice for the 
purpose of strengthening the systems that serve children, families, and communities. In pursuit of that 
agenda, Chapin Hall maintains a diverse research portfolio that is methodologically varied and theoretically 
grounded. In addition to the Chapin Hall’s core research program, the Center supports knowledge 
translation. Of particular interest, Chapin Hall’s Center for State Child Welfare Data supports training and 
technical assistance in advanced analytics, continuous quality improvement, and evaluation design. 
Importantly, Chapin Hall is affiliated with the University of Chicago. The affiliation gives Chapin Hall staff 
access to leading methodologists and statisticians, extensive library resources, and a robust computing 
environment. Finally, staff of the Center for State Child Welfare Data have extensive experience working 
with state administrative data, including Allegheny’s data warehouse. Our experience includes the 
development and maintenance of integrated data systems similar to the used to support the Predictive Risk 
Model. 

Staff at Chapin Hall and the Center for State Child Welfare data have a long history of designing 
evaluations of public child welfare programs for federal, state and local government agencies. The work 
has been both basic and applied in nature. The Big Data revolution in child welfare was started at Chapin 
Hall in the early 1980s when the first longitudinal event history files were developed from Illinois child and 
family tracking data. With those data, staff at Chapin Hall went on to pioneer the use of event history 
analysis for purposes of understanding what happens to children in contact with the child welfare system. 
Early studies included multistate comparisons of child outcomes (e.g., length of stay, reentry, placement 
stability, the role of local ecology vis a vis the incidence of child abuse and neglect and the demand for 
foster care. Much of that work has had a direct impact on the outcomes measures adopted by the U. S. 
Department of Health and Human Services as part of the Child and Family Service Reviews. The Center’s 
basic research carries over into applied settings, with an emphasis on both summative and formative 
evaluations of service innovation. Chapin Hall researchers have designed experimental, quasi- 
experimental, and observational evaluation studies of innovative child welfare programs including family 
preservation, the implementation of evidence-based interventions, and decision-making using family/group 
decision models. 

Chapin Hall’s diverse experience is particularly well suited for the PRM evaluation. First and foremost, we 
have extensive experience with the DHS warehouse data having used that data to build longitudinal event- 
based files for the evaluation of the county’s system of care, wraparound, and family support centers. The 
principles behind those designs are particularly relevant in the case of the PRM. Predictive risk modeling 
relies heavily on what happens next in the case trajectory of a child who has been screened in (or not) 
inasmuch as false positives and negatives are determined on the basis of subsequent events. Chapin Hall 
staff are already in position of the computer algorithms needed to organize the analytical files. With those 
files and other similar files, Chapin Hall researchers have expanded the use of multi-level, discrete time 
models that are especially well-suited to understanding how decision processes and their consequences 
are affected by the characteristics of children, caregivers, providers, and contextual factors. Finally, Chapin 
Hall has considerable experience carrying out and supervising research that examines the implementation 
of programs and other innovations that target decision-making processes. Working in settings as diverse as 
Tennessee, New York City, and Michigan, our portfolio includes research that links data collected 
qualitatively with administrative data at the worker, child/family, and agency level so as to understand how 
workers use time and whether time use affects decision making and outcomes. 

3.1.1.2 Kempe Center for the Prevention and Treatment of Child Abuse and Neglect 

For more than 40 years, the Kempe Center has promoted understanding, knowledge, and best practices to 
prevent and treat child abuse and neglect locally, nationally, and internationally. Kempe fulfills its mission 
through research and evaluation; training, technical assistance, and education; and service delivery, 
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advocacy, and consultation. Kempe is administratively located within the Department of Pediatrics at the 
University of Colorado Anschutz Medical Campus in Aurora, Colorado. Kempe’s 100 faculty and staff are 
experienced in designing, implementing, evaluating, and disseminating a wide range of evidence-based 
programs and initiatives (e.g., differential response, trauma-focused mental health treatment, family group 
decision making, risk and safety assessment, and community-based child abuse prevention) to prevent and 
treat child abuse and neglect as well as improve the health and well-being of children and families. When 
several staff members from the American Humane Association joined the faculty of Kempe in 2012, their 
addition made Kempe distinctive among similar organizations in the United States and internationally by 
increasing its breadth of expertise related to child welfare practice, policy, systems, and research. 

With respect to the work proposed herein, Kempe faculty are known for their groundbreaking research 
using an ecological approach to understanding factors affecting child welfare staff decision making and 
outcomes. This experience includes 1) the identification of case factors that influence the likelihood that 
child maltreatment will recur or that families and/or children will be the subject of repeat referral or re¬ 
reports; 2) the identification of agency, worker, and case characteristics’ that influence decision making 
practices, 3) the impact of policies, practices, and demographics on states’ child maltreatment recurrence 
rates, and 4) using research findings to develop safety and risk assessment tools, training, and policies. 
These efforts have yielded tools and models that predict variations in outcomes from the individual to the 
aggregate level such as counties or states. The experience entailed will be leveraged to inform the design 
and implementation of the evaluation in ways that are both highly specific to the evaluation needs of the 
Allegheny PRM evaluation, but also will reflect lessons learned from a range of prior evaluations of public 
child protection systems. 

Kempe staff are highly accustomed to working with the myriad professionals across the multiple systems 
that work to protect children and support vulnerable families. Kempe staff have conducted focus groups, 
interviews, and surveys with a spectrum of public and private child welfare staff (administrators, program 
directors, supervisors, caseworkers, information technology staff and analysts, administrative staff, and 
volunteers) at federal, state, and county levels. Given the interconnection between the formal child welfare 
system and other human service systems, the Kempe team has also acquired significant experience 
gathering information from and involving other stakeholders, including parents/caregivers and 
representatives from mental health, substance abuse, prevention, juvenile justice, courts, and 
developmental disabilities. Kempe staff will draw on these experiences as they collaborate with Chapin Hall 
and Joyce James Consulting team members to evaluate the decision support and predictive analytic tools. 
The expertise and accomplishments of Kempe’s faculty reflect a deep commitment to conducting research 
and translating it into practice. Moreover, the development and application of these and other analyses 
have involved the specification of data extracts from child welfare, mental health, heath, benefits and other 
information systems in many states, construction of event data, and the use of a range of analytic 
procedures. Kempe staff are well-versed in a variety of methodological approaches and have led studies 
involving randomized control trials, propensity-score comparisons and other quasi-experimental designs. 
Other specific program and system evaluations conducted by Kempe staff include the impact of differential 
response, family group decision making, and factors influencing child welfare disparities. The depth and 
breadth of their experiences reflects a firm commitment to the use of evaluation to inform policy and 
practice decisions, advance the understanding of workforce development, develop models of agency 
finance and costs, improve outcomes aimed at strengthening child welfare agencies. 

3.1.1.3 Joyce James Consulting, Inc. (JJC) 

Joyce James Consulting (JJC), a minority woman owned firm, has had a number of years’ experience in 
conducting process evaluations that have included key informant interviews, focus groups and training in a 
number of states that include New York, Pennsylvania, Texas, Indiana and Oklahoma. In this context she 
has worked with a number of cross-system agencies. Joyce James, the owner and principal in JJC, is a 
nationally recognized expert in racial equity, social injustices, and system transformations all within a 
backdrop of decision-making. With a professional career spanning more than three decades, Ms. James 
provides consulting services to organizations and institutions at various levels in both the public and private 
sectors. In addition; as reflected on the enclosed resume, Ms. James has had many successful 
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experiences in supporting the strategic goals and objectives of child welfare systems, local education 
agencies, colleges, and universities. 

Ms. James most recently served as the Associate Deputy Executive Commissioner for the Center for 
Elimination of Disproportionality and Disparities (Center) at the Texas Health and Human Services 
Commission (HHSC). The Center was created by former HHSC Executive Commissioner Thomas Suehs. 
Ms. James was appointed to the leadership role within the Center and HHSC in September 2010 in 
recognition of Ms. James' record of strong and effective leadership and decision-making throughout her 
career and a desire to expand the Texas Model for Addressing Disproportionality and Disparities— 
developed under her leadership—to all Health and Human Services (HHS) agencies and programs. She 
was principle investigator on a number of national and local grants which had both impact and process 
evaluation components. These included funding from the Children’s Justice Act through the Texas 
Supreme Court's Permanent Commission on Children Youth and Families where pre-post surveys yielded 
both quantitative and qualitative evaluation data regarding the impact of training on the decision-making of 
school personnel as reporters of child abuse and neglect, eight grants from the Federal Office of Minority 
Health through the Center's role as State Officer of Minority Health, and funding through Texas’ state 
Medicaid office to study the immunization process for the poor where focus groups and interviews were 
conducted and surveys were administered. 

Prior to becoming the Associate Deputy Executive Commissioner at HHSC, Ms. James was the Deputy 
Commissioner for the Texas Department of Family & Protective Services (DFPS). At DFPS, Ms. James 
provided leadership for the Center for Learning and Organizational Effectiveness with responsibility for 
developing and implementing training for approximately 11,000 DFPS from entry to executive level. The 
four years before that, Ms. James was the head administrator and decision-maker for Child Protective 
Services in Texas where she championed impact and process evaluations that helped solidify the view of 
caseworkers as decision-makers. She published a number of articles with others on the Decision-making 
Ecology that resulted from these efforts. Three interventions that were funded by the legislature were 
evaluated from and impact and process perspective. They were Family Group Decision-Making, The 
Relative Care Assistance Program and the Strengthening Families Initiative. A fourth, funded through 
Casey Family Programs and the American Humane Association, used a Decision-Making Ecology 
approach, such as that suggested here. The study used a multi-level model based of with data from 
surveys (process) linked to administrative data containing actual worker decisions. 

3.1.2 Information about personnel 
3.1.2.1 Chapin Hall Center for Children 

Fred Wulczyn, Ph.D., is a senior research fellow at Chapin Hall and founding director of the Center for 
State Child Welfare Data. Trained at the University of Chicago, Dr. Wulczyn has focused his work on the 
use of electronic or computerized case records for purposes of research and evaluation. He designed the 
original longitudinal foster care placement files, now in use at Chapin Hall since 1980. In the process of 
continually updating the designs he developed, Dr. Wulczyn has integrated maltreatment, public 
assistance, Medicaid, birth records, education, and mental records with placement data to provide a 
comprehensive view service involvement. In addition to his work designing databases, Dr. Wulczyn has 
extensive experience analyzing those data. Geographic information, hierarchical modeling, event history 
models, and simulation fall within the range of his skills. Having worked as a senior government official in 
New York State, and later as a consultant to the U.S. Department of Health and Human Services during the 
Obama Administration, Dr. Wulczyn brings a deep appreciation for decision-making under conditions of 
uncertainty. 

Jianyu Wang, Ph.D., is the data base administrator for the Center for State Child Welfare Data. In that role 
he oversees data management, which includes data receipt, data processing, and backup. He also 
maintains the Data Center’s online web portal. He too has experience creating longitudinal data records 
from state SACWIS and SACWIS-like systems. He performed much of the file manipulation with the 
Allegheny data when it first came to Chapin Hall. He also just completed a project in Delaware that called 
for creating an event history file from CPS and placement data. 
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3.1.2.2 Kempe Center for the Prevention and Treatment of Child Abuse and Neglect 

John Fluke, PhD, is Associate Director of System Research and Evaluation at the Kempe Center for the 
Prevention and Treatment of Child Abuse and Neglect and Associate Professor at the University of 
Colorado School of Medicine. With a career focused on the areas of child welfare and mental health 
services for children, he is internationally recognized as a researcher specializing in assessing and 
analyzing decision making in human services delivery systems. His contributions to Decision Making 
Ecology are widely recognized. He is also active in the area of national child maltreatment data collection 
systems and analysis and has worked with data collection programs in the Balkans, Canada, Saudi Arabia, 
the US, and for UNICEF. He is also known for his innovative and informative research and evaluation work 
in the areas of child maltreatment prevalence, child welfare administrative data analysis, workload and 
costing, and performance and outcome measurement for children and family services. 

Dana Hollinshead, PhD, is an Assistant Research Professor at the Kempe Center for the Prevention and 
Treatment of Child Abuse and Neglect, University of Colorado School of Medicine, Denver, CO. Dr. 
Hollinshead is a graduate of the Heller School of Social Policy and Public Management at Brandeis 
University. She has extensive child welfare project management, research, program evaluation, and policy 
analysis experience utilizing advanced quantitative and qualitative data collection, analysis. Owing to her 
experience with in working with child welfare agencies in states and having worked with the children's 
bureau Dr. Hollinshead has gained insight into federal, state, and local government perspectives on child 
welfare programs, policies, practices, administrative data, performance measurement, program 
improvement, and system reform efforts. Her expertise extends to examining a comprehensive array of 
factors affecting child welfare outcomes including decision-making. Dr. Hollinshead is also highly 
experienced in disseminating research and evaluation results and has made many presentations, 
conducted trainings, provided technical assistance, and prepared many reports and publications. 

3.1.2.3 Joyce James Consulting, Inc. 

Joyce James, LMSW, the owner and principal in JJC, is a nationally recognized expert in racial equity, 
social injustices, and system transformations all within a backdrop of decision-making. With a professional 
career spanning more than three decades, Ms. James provides consulting services to organizations and 
institutions at various levels in both the public and private sectors. In addition; as reflected on the enclosed 
resume, Ms. James has had many successful experiences in supporting the strategic goals and objectives 
of child welfare systems, local education agencies, colleges, and universities. Ms. James most recently 
served as the Associate Deputy Executive Commissioner for the Center for Elimination of Disproportionality 
and Disparities (Center) at the Texas Health and Human Services Commission (HHSC). Prior to becoming 
the Associate Deputy Executive Commissioner at HHSC, Ms. James was the Deputy Commissioner for the 
Texas Department of Family & Protective Services (DFPS). At DFPS, Ms. James provided leadership for 
the Center for Learning and Organizational Effectiveness with responsibility for developing and 
implementing training for approximately 11,000 DFPS from entry to executive level. The four years before 
that, Ms. James was the head administrator and decision-maker for Child Protective Services in Texas 
where she championed impact and process evaluations that helped solidify the view of caseworkers as 
decision-makers. She published a number of articles with others on the Decision-making Ecology that 
resulted from these efforts. 

Donald Baumann, Ph.D. has over thirty years' experience designing and managing large-scale research 
projects as both a Principal Investigator and as a Project Director. He has been on the faculties of the 
University of Texas, Trinity University and Saint Edwards University. While at the University of Texas he 
began his exploration of decision-making with consequences while working with the police and Travis 
County Prosecutors Office. He has directed numerous national multi-year research and evaluation projects 
over the years, most of which have been focused on understanding decision-making or finding solutions to 
augment that process (e.g., risk assessment). He is retired from the Texas Department of Family and 
Protective Services where he was head of the Evaluation Section of Child Protective Services. While there 
for twenty years he helped develop The Decision-Making Ecology which has been empirically applied to 
caseworker burnout and performance, the intake process, the substantiation decision, the decision to place 
children into care and the decision to reunify children with their parents. The model looks at the context of 
decision-making as a process, in relation true positives and negatives as well as false positives and 
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negatives as outcomes of these decisions. He currently teaches at St. Edwards University and is a 
Research Associate with Joyce James Consulting, LCC. He has written over eighty reports, articles and 
book chapters. 

Windy Hill has a 20-year career in executive leadership and management has included successful 
experience in public administration, nonprofit management, community organizing and development and 
consulting at the grassroots, state and national levels. Windy is currently an Associate Racial Equity 
Consultant with Joyce James Consulting, LLC; where she provides qualitative data acquisition and 
analysis, consultation, professional development training and community organizing through processes that 
create momentum and direction for improving racial equity within systems and institutions. Windy served 
most recently as a senior policy and program specialist in the Texas Health and Human Services 
Commission (HHSC) Center for Elimination of Disproportionality and Disparities (CEDD) where she 
performed complex activities and administrative functions that supported the HHSC Associate Deputy 
Executive Commissioner’s management and oversight of the CEDD. In this role, she spearheaded a 
number of decision-making projects. 

3.2 Approach to Process Evaluation 

3.2.1 Background and key research questions 

The PRM is designed to improve decisions. Having said that, the PRM does not in fact make the decision. 
Rather, the PRM brings forward into the decision making process information about the active case that is 
meant to be used by the screener as he or she makes the screen in/screen out decision. Because the final 
decision remains in the hands of the screener, understanding how decisions are made, why decisions are 
made, and the role PRM results played are key components of the evaluation. Specifically, if the PRM 
implementation gives screeners the authority to override the PRM results, then knowing when and why 
screeners exercise that authority is vitally important. At the same time, how results from the PRM are 
presented to the worker raises questions of implementation and implementation fidelity. 

With these challenges in mind, we have identified three basic goals of the process study. The first is to 
better understand implementation model and its impact on how the PRM is used. Questions tied to this 
goal include: 

1. How workers interpret and incorporate different PRM displays of the same data into their 
assessments? 

2. How much confidence do DHS Frontline workers have in the information? 

3. How easily understood and accessible is the PRM Tool? 

4. Are the results of the PRM Tool displays those desired? 

The second goal has to do with how the PRM changes the decision-making process itself. Because the 
PRM regularizes access to information connected to risk and the assessment of risk, workers will likely 
react to the ease with which otherwise difficult-to-access information is more available. However, the 
presentation of the information has to fit with usual work routines that may vary by screener and length of 
time on the job. To address these issues, our evaluation plan includes these additional questions: 

1. How easily understood and accessible is the PRM Tool? 

2. Are the results of the PRM display easy to comprehend? 

3. Does the PRM change the business process and/or length of time to make a decision? 

4. Has the PRM helped or hindered the early identification and engagement of families? 

5. How the partners (including DHS leadership and the Juvenile Section of the Family Division of eh 
Allegheny County Court of Common Pleas) view and value the PRM Tool? 

The third goal of the process evaluation is to explain how context influenced the decisions made. For this 
goal, the key research questions are: 

1. How do case factors (e.g., race, poverty etc.) affect the decisions of workers? 

2. How do features of the individual decision-maker (e.g., worker orientation - child safety vs. family 
preservation, experience and skill sets, etc.) affect the decisions of workers? 
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3. How do workload, resources, organizational support, supervision, service availability affect the 
decisions of workers, including their willingness to override the PRM? 

4. How do external factors (e.g., highly publicized child fatalities, legislative action, etc.) affect worker 
decisions? 

3.2.2 Theoretical Framework 

Both the process and the impact evaluation are guided by the DME model. For the process study the key 
DME insight has to do with the fact that decisions are not made in isolation. Attributes of workers, cases, 
organizations, communities, and history define the context in which decisions are made. Risk assessment 
and other tools designed to use the past to better forecast the future are but one input in a highly diverse, 
ever changing context. As such, use of the PRM is subject to how the decision-maker sees the utility of the 
PRM relative to all of the other influences that ultimately shape a given decision. Implementation and 
success of the PRM generally depend on the worker’s judgment. 

To the extent the escalation/de-escalation framework is used to classify decision outcomes, results of a 
decision classification allows us the opportunity to adjust questions based on where the decision falls in the 
contingency table. True positives and true negatives provide an opportunity to discuss the role of the PRM 
relative to other influences. The same holds for false positives and false negatives. In the end, the goal of 
process study is to understand how the PRM fits within decision-making routines, the extent to which the 
PRM trumps other contextual influences including the worker’s own judgment, and establish how fidelity to 
PRM implementation model affects the two former questions. 

3.2.3 Methods and Data Collection Strategy 

There are three data collections strategies proposed: surveys, key informant interviews and focus groups. 
Instruments for both data collections methods will adhere to the theoretical framework described earlier: 
what is the context of the decision, how well integrated is the PRM in decision-making routines, and do 
different implementation models help or hinder the use of the PRM. 

3.2.3.1 Prospective Screener Surveys 

Because the data collection strategy for the process study depends to some extent on how the decision is 
ultimately classified, we are concerned that the process evaluation study is overly reliant on recollection, 
which is subject to certain biases. For that reason, in consultation with DHS, we are proposing both 
retrospective and prospective surveys. For the prospective survey, we will develop a brief check list of 
items screeners can fill out at the time they are making the decisions. Our approach would randomly select 
a time of day and a day of the week plus one screener from a roster provided by DHS. At that randomly 
selected time, the randomly selected screener would be asked to complete a five-minute on-line survey 
designed to quickly extract key characteristics of the case and the decision-making context. Pending a 
feasibility discussion with DHS staff, the approach provides us within-the-moment data that will more 
reliably identify the contingencies that were being weighed as the decisions were being made. We are, of 
course, concerned with whether this data collection adds to screen burden; moving forward would require 
careful coordination should DHS decide the advantages of having these data outweighed the 
disadvantages. These surveys would run throughout the evaluation period. Results would use a part of 
the formative evaluation reports. Pending discussion with DHS regarding the PRM implementation sample 
size decisions are pending. 

3.2.3.2 Staff Surveys 

Staff surveys will be used to better understand the context of decision-making. These surveys will include 
more in depth questions than the prospective surveys proposed earlier. Typically, the surveys are 
administered to workers and supervisors. The surveys will be designed so that we can determine which 
sets of factors exert the most influence on decision making. Survey results will be used to better 
understand and modify (if needed) features of the PRM. The following lists of four contextual dimensions 
provide a general orientation to the survey construction. 

1. Case Factors: These can be obtained through a combination of a surveys and administrative data. 
Data collected include pattern of maltreatment (e.g., severity, chronicity and trend), child 
vulnerability (e.g., age), caretaker capability (e.g., parenting skills), home and social environment 
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(e.g., violence), quality of care (e.g., connection to child) response to CPS (e.g., deception), and 
disparities (e.g., race and poverty). 

2. Individual (i.e., screener) Factors: These typically are obtained through survey data or through 
access to HR records. They include experience, skills (e.g., interpersonal, case and 
administrative), concern over personal liability, and worker orientation (e.g., child safety vs. family 
preservation). 

3. Organizational Factors: These can be obtained through survey and/or administrative data. These 
data include workload, resources (e.g., service availability), organizational support, supervisor 
support, administrative distractions (e.g., unnecessary interruptions) and policy changes. 

4. External Factors: These can be obtained through surveys and/or administrative data. They include 
changes in the law, critical events (e.g., highly publicized child fatalities or injuries), and changes in 
funding levels. 

5. Implementation: In addition to DME inspired questions we will use the surveys as an opportunity to 
learn about implementation with a focus on ways workers seeing the PRM as an aid to their 
decision-making. 

We propose two cycles of surveys - one pre and another post implementation in order gauge if and 
decision making processes change with the introduction of the PRM tool. The pre-test is dependent on the 
implementation plan and whether there is enough time between contract award and the onset of the PRM 
trial. To facilitate this process, the project team has access to surveys used in other DME studies that have 
been validated (Dettlaff, Christopher Graham, Holzman, Baumann, & Fluke, 2015)and are set up in both 
Survey Monkey and Qualtrix. Through each data collection opportunity and the analysis of those data, we 
will work with DHS to ascertain how the collected data can be used to support the formative aspects of the 
evaluation. 

3.2.3.3 Focus Groups and Key Informant interviews 

Focus group participants will include (1) screeners and other PRM users, (2) Allegheny DHS leadership, 
and (3) the broader set of community partners (e.g., family court personnel, agency social workers). Focus 
group invitations will be determined in consultation with Allegheny County DHS and AUT Enterprises Ltd. 

Post implementation focus groups will be conducted with each of the above groups over a three-day period 
after implantation of the PRM tool at a date and time convenient for participants. The three focus groups 
will occur on day one and two. Day three will be reserved for those who were unable to attend the first two 
days or individual interviews as needed. The focus groups will be conversational in tone; focus group 
leaders will have in hand results from surveys, and preliminary results from the AUC and DTM. Where 
appropriate, we will use the focus groups to deepen our understanding of those findings. 

3.2.4 Analytical Strategy 

Focus group and interview questions will be recorded and constructs from the answers to questions 
extracted. Data will be entered into NVivo or Atlas.TI. Coding themes will be developed from these data. 
Coding schemes will be tested for reliability. These data will form the basis of a qualitative study with the 
narrative organized around the DME and escalation/de-escalation frameworks. 

Survey data will be compiled directly from the on-line surveys into a SAS, SPSS, or STATA datafile to 
facilitate analysis. These data will be then merged with administrative data by way of a link to worker IDs, if 
available. As the data in these files are organized around the DME/escalation/de-escalation framework, the 
collected data will be used to understand decision reversals and the role of context, worker characteristics, 
case characteristics. The qualitative data from the process study will be used bring draw the nuances 
embedded in the quantitative data. 

3.2.5 Report 

We are keenly aware of the fact the PRM models, as applied to child protection decisions, are largely 
untested. Because the affected decisions are profoundly important, a formative evaluation meant to guide 
implementation is unusually important. For this reason, we will work with DHS and the model team, from 
the outset, for the purpose of devising a reporting schedule that provides timely feedback to DHS. The main 
intent is to provide guidance as to whether false negative rates are rising. Other changes, including 
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feedback from staff (based on the process study) that affect implementation fidelity are as important. We 
think routine integrated reporting from both the impact and process is key to a successful PRM trial. 

With regard to a final report, we propose delivering a draft report four weeks prior to the end of the PRM 
trial. The final report would then be delivered two weeks after receipt of comments from DHS and the PRM 
team. 

3.3 Approach to Project Management 

3.3.1 Team Collaboration 

The partnership formed for the PRM evaluation is based on longstanding, successful working relationships. 
Chapin Hall and the Kempe Center are partners on evaluations involving the Tennessee IV-E waiver 
program and the Texas foster care redesign. In both cases, as proposed for the PRM evaluation, Chapin 
Hall serves as the prime with a subcontract to the Kempe Center. In addition, Dr. Fluke from Kempe has 
worked closely with staff from Joyce James Consulting (JJC), having published articles with Dr. Baumann 
covering a wide range of issues connected to decision-making. During her tenure with the Texas 
Department of Family and Protective Services, Joyce James worked with Dr. Fluke on issues related to 
racial disparities in the child welfare system. 

The approach to collaboration is based on clear delineation of responsibility for work products. Chapin Hall 
will assume overall management for fiduciary and substantive matters. With regard to work products, 

Chapin Hall will take lead responsibility for data extracts from the DHS warehouse. Those responsibilities 
include data acquisition, management, and security. Development of analytical files will also fall to Chapin 
Hall, but in conjunction with Allegheny staff, the PRM research team, and other members of the evaluation 
team. The design focus is on operationalization of key independent and dependent variables relative to 
what success means (e.g., the sensitivity and specificity of the PRM tool). These data will form the basis of 
the impact evaluation. All partners will be involved in data analysis and interpretation: Chapin Hall will take 
the lead with regard to selecting appropriate statistical models; Kempe and JJC will work with Chapin Hall 
to develop the conceptual framework, with a specific focus on the decision-making ecology. Interim and 
final work products will be developed jointly. 

3.3.2 Research Team 

With regard to impact, work with the research team will focus on model performance. One aim of this work 
is understanding how the model was constructed and any preliminary results so that the model structure 
can be incorporated into the impact evaluation. We will also want to know whether stratification was used. 
All members of the evaluation team will participate in this discussion. We expect initial discussions with the 
research team to take place within two weeks of award and again after an initial specification of the impact 
analysis. Thereafter, bimonthly meetings will be scheduled so that progress and preliminary results can be 
shared. Review of the draft report will occur in month 11. In the event our preliminary analysis warrants 
more frequent updates, we are prepared to support additional meetings with the research team. 

3.3.3 Allegheny Team 

Our collaboration with Allegheny team follows the model established for the research team. We see 
meetings with the research team and the Allegheny team as joint meetings so that all members of the 
collaboration are intimately involved in the decision-making. As implementation of the PRM is in the hands 
the Allegheny team, we expect to organize the meetings but co-facilitate the meetings with the Allegheny 
team. Meeting agendas will be developed jointly will all partners (Evaluation Team, Research Team and the 
Allegheny Team. 

With regard to implementation of the PRM, we understand that the PRM is designed to improve decision¬ 
making relative what the decisions would have been in the absence of decision-support technology. We 
also understand that because the PRM addresses predictions having to do with child safety, the accuracy, 
specificity, sensitivity, and precision of the PRM are particularly important. For that reason, in conjunction 
with the research team, we propose working with the Allegheny Team to establish a threshold below which 
there is sufficient evidence to suggest the model is not working as intended. In the event the threshold is 
reached we will convene all stakeholders to undertake a formative review of the model with the aim of 
determining whether the model should be pulled from the operational context. As part of the decision- 
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making process, we will work with the Allegheny team to develop an interim report schedule so as to inform 
the process. 

3.4 Requirements/limitations/challenges 

The principal requirements/limitations/challenges have to do with the ability and willingness of participants 
to participate in the focus groups, interviews and surveys. Workers may want indemnification and while not 
the responsibility of the evaluators, this may affect the implementation timeline. We will work in 
collaboration with the Allegheny Team to help insure as much participation as possible. Additionally, the 
low number of intake workers is a challenge. To some degree this is handled by repeated measures of 
decision-making and the multi-level structural modeling process. 

Studies that have tested the DME suggest that context matters. We are acutely aware of the fact that 
circumstances beyond the control of anyone connected with the PRM trial could substantially alter the form 
of the trial. Again, with the historical data we can judge the likely impact of unpredicted events external to 
the project. 

We expect the effort needed to secure IRB approval will be significant. Our team, with former IRB 
members, has the experience to secure the necessary IRB approvals. That said, among the challenges, 
the time needed to secure IRB approval could affect the start of the project and the evaluation. 

3.5 Validity/strength of results 

The DME provides the basis for understanding sources of variability in decision-making. Typically, these 
analyses are performed at the worker or supervisory level to obtain estimates of the probability that staff or 
work units are likely to take a decision action (e.g., refer a case for a response) and are a proxy for the 
decision thresholds. These probabilities are in turn be associated with characteristics of the individual or 
group members, organizational variables and external factors (values of management, perceptions of 
liability, etc.) and risk adjusted for the case characteristics. These analyses are designed to help determine 
which sets of factors that are not case related exert the most influence on decision making. 

Over time during implementation, we would anticipate that successful identification and modification of 
leverage points aimed at reducing variability would result in changes to the statistical relationships between 
factors in the DME and decision thresholds. For example, if workload were determined to have a strong 
relationship to variability and workload was modified; then we would expect that workload would no longer 
contribute to variability. The underlying idea is to reduce variability and thus improve the reliability of actual 
decision-making behavior in the system. 

Although context and other influences contribute to decision-making, individuals and their predispositions 
play a key role. Over time, with transfer in and out of the unit, the population of screeners and supervisors 
is likely to change. As a consequence, our evaluation is dependent on the workforce in place now. As that 
changes the net worker influence, as either an aggregate or as individual, is likely to shift. This is true in 
any unit, but with safety decisions it is important to closely watch the decisions as they are made. We 
believe that our evaluation strategy establishes a model for monitoring decision-making, which will bring 
DHS longer term benefits, whether it adopts a PRM or not. 

References: 

Baumann, D., Dalgleish, L., Fluke, 1, & Kern, FI. (2011). The Decision-Making Ecology (pp. 1-15). 
Washington, DC: American Flumane Association. 

Dettlaff, A. J., Christopher Graham, J., Flolzman, J., Baumann, D. J., & Fluke, J. D. (2015). Development of 
an Instrument to Understand the Child Protective Services Decision-Making Process, with a Focus on 
Placement Decisions. Child Abuse & Neglect, 49, 24-34. 
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4 Budget and Budget Narrative 

4.1 Budget Narrative 

As provided the budget covers the contribution of each partner organization: Kempe Center and Joyce 
James Consulting are listed as subcontracts. As the prime contractor, Chapin Hall assumes lead fiduciary 
and substantive responsibility. More generally, work products are divided in this way: Chapin Hall will take 
the lead on the impact study with support from Kemp and Joyce James Consulting. For the process study, 
Joyce James Consulting will lead. Kempe Center is allocated across phases of the evaluation. Specific 
allocation by organization follows. 

Chapin Hall : 

Principal Investigator Wulczyn will have overall responsibility for the design of the impact study. He will 
also consult on the process evaluation. In that capacity, he will oversee that statistical analysis. He will 
participate in project management meetings with the other partners, DHS, and the PRM. He will review 
reports prior to submission and be actively engaged in the review process. He will also work closely with 
the project leads Fluke, James, and Baumann. He will oversee the IRB application, through the University 
of Chicago IRB. Dr. Wulczyn is budgeted at 0.08 FTE for the length of this project. 

Lijun Chen, Senior Researcher, will lead the data analysis team from Chapin Hall. Data analysis includes 
aligning data with the administrative so that decisions made and decision outcomes align DHS/PRM 
definitions. He will develop models for baseline and pre/post assessments of decision outcomes. Dr. Chen 
will assist with the IRB application. Dr. Chen is budgeted at 0.16 FTE for the length of this project. 

Jianyu Wang, Sr. Database Administrator, will manage data transfers, data management, and data 
security at Chapin Hall. We anticipate regular data updates (i.e., on a bimonthly basis in order to support 
the formative evaluation, as proposed. In coordination with Dr. Chen, Dr. Wang will also create linkage files 
based on survey and other data, as the opportunity arises. Dr. Wang is budgeted at 0.08 FTE for the length 
of this project. 

Fringe Benefits and Paid Time Off: Paid Time Off (PTO) is an equitable way to distribute paid time off 
earned during the course of a research project. Based on a three-year average of historical data of staff 
paid time off earned and used, Chapin Hall has calculated a rate of 14% to be applied to salary expense. 
The PTO expense is calculated as the product of the PTO rate and actual hourly expense budgeted. Fringe 
benefits (25%) are applied to the sumio^a|ary and PTO expense. 

Personnel costs for Chapin Hall total 

Indirect Costs Chapin Hall Center for Children at the University of Chicago does not have a federally- 
negotiated indirect cost rate agreement at this time but is currently in the midst of negotiations. Indirect 
costs are currently estimated at 55% and are applied to total direct costs, for a total of $31,568. 

Subcontractors: 


Kempe Center at the University of Colorado : 

For the proposed effort Kempe faculty will work in the following areas: evaluation design, coordination and 
analysis of survey data, analysis of PRM model performance including work with administrative data, and 
utilization and dissemination of results and findings. The budget lays out the anticipated contributions in 
terms of percent of FTE allocated to the project with 5% allocated for the Kempe PI, 20% for the research 
associate/statistician, and 20% for a professional research assistant whose role will be to coordinate and 
maintain survey and other project data. FTE allocations are budgeted based on actual salaries and 28% 
fringe for a total of $40,812. Four trips to Allegheny County are budgeted at an estimated cost of $1,250 per 
trip with the expectation that these will be required for project planning and dissemination. Finally, the 
standard 25% overhead rate required by the contracting arm of the University of Colorado School of 
l\^gne - University Physicians Inc. (UPI) is included. The total Kempe subcontract is budgeted at 

Co-investigator John Fluke will assist with both the impact and process study. He will assume a lead 
responsibility for translating the PRM decision structure into a data structure, in conjunction with DHS, the 
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PRM team, and other evaluation partners. In addition, he will work with Joyce James Consulting to 
operationalize the DME theory into a measurement model for incorporation into the survey and focus group 
instruments. During data analysis phases, he will assist with preparing reports (formative and summative. 
To that end, he will prepare analysis, focusing on the AUC models. He will also support the DTM analysis. 
Dana Hollinshead will work across partner organizations, with responsibility for survey and focus group 
instruments for use during the process study. Her specific focus will be alignment of the instruments with 
the DME theory and pretesting of instruments where pretests are called for. During data analysis, she will 
focus on narrative integration of the process study with the impact study. She will also contribute to the IRB 
protocol and participate in project meetings. 

Joyce James Consulting : 

For the proposed effort Joyce James Consulting will work in the following areas: Discuss with Allegheny 
CYF staff and AUT Enterprises LTD to review and refine focus group and survey questions, as well as plan 
and organize focus groups that will be conducted by Joyce James Consulting. Surveys will be 
administered, and data collected from the surveys and focus groups analyzed. Her group will also assist 
with linking the survey data to the administrative decision-making data and assist with analysis of the 
disparity data. Finally the group will write the process evaluation component of the report. The budget lays 
out the anticipated contributions in terms of dollars per hour and number of hours allocated to the project 
with 107 hours allocated for the Joyce James Consulting PI, and 94 hours and 110 hours for the Research 
Associates respectively. Allocations are budgeted based on actual hourly rates and 0% fringe for a total of 
$33,700. Two trips for two persons (4 trips) to Allegheny County are budgeted at an estimated cost of 
$1,500 per trip with the expectation that these will be required for project planning and dissemination. 
Finally, the 18% overhead rate is that cfwaecyby Joyce James Consulting. The total Joyce James 
Consulting subcontract is budgeted at $^. 

Joyce James, principal of Joyce James Consulting, will co-lead the process study with Donn Baumann. In 
that capacity, James will have lead responsibility for devising and conducting the process study, with 
emphasis on decision-making in an organizational context. She will work with Windy Hill conducting focus 
groups and interpreting data from surveys. During the data analysis/interpretation phase she will focus on 
disparities and the role of context on decision-making as those issues present themselves in the results. 
She will also guide summative reports, insuring relevance for DHS leadership. She will also participate in 
all project meetings with DHS, the PRM team, and project partners. 

Donn Baumann, co-principal for Joyce James Consulting, will work across the impact and process studies, 
with a focus on integrating the DME across data collection and data analysis phases. He will have lead 
responsibility for the qualitative data collected during the process study, with an emphasis on instrument 
development. He will also contribute project meetings with DHS and preparation of the IRB protocol. 

Wendy Hill will assist in all phases of the process study, with an emphasis on the surveys and focus 
groups instruments, working collaboratively with Dana Hollinshead. With Joyce James, Hill will conduct 
focus groups and prepare summary findings from the qualitative components of the study. With other 
members of the team, she will contribute to the IRB protocol. 
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4.2 Financial Audit 


Please see attachment 6.5 following the CVs/Resumes for Chapin Hall’s audited financial statements and 
independent auditor’s reports. 
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5.1 Chapin Hall 


Tennessee Department of Children’s Services 

Commissioner Bonnie Hommrich 

Cordell Hull Building 
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Nashville, TN 37243 

(615) 532-3591 

bonnie.hommrich@state.tn.us 


New York City Administration for Children’s Services 
Associate Commissioner Andrew White 
150 William St. 
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6 Attachments 

6.1 Cover Page 

ALLEGHENY COUNTY 
DEPARTMENT OF HUMAN SERVICES 

PROPOSAL COVER PAGE 

SOLICITATION: 

Evaluation of a Predictive Risk Modeling Tool for Improving the Decisions of Child Welfare Workers 


Contact Information 

ORGANIZATION OR INDIVIDUAL: 

Chapin Hall Center for Children at the University of Chicago 

AUTHORIZED REPRESENTATIVE: 


Name: Bryan Samuels 

Title: Executive Director 

ADDRESS: 

1313 East 60 th Street 

Chicago, IL 60637 


TELEPHONE #: (773) 256-5116 

FAX #: (773) 256-5316 

E-MAIL ADDRESS: BSamuels@chapinhall.org 

WEBSITE: www.chapinhall.org 



Proposal Information 


DATE SUBMITTED: 01/04/2016 
AMOUNT REQUESTED: $193,074 

^PROPOSAL ABSTRACT: Our proposal covers impact and process evaluations. Evaluation partners are 
the Kempe Center at the University of Colorado School of Medicine and James Joyce Consulting. The 
impact study uses the Decision-Making Ecology and a contingency matrix to classify decisions and 
whether predictions are reversed in time. Discrete time models are proposed for a pretest/posttest 
analysis of whether prediction errors change during the PRM from baseline. The process study 
focuses on implementation. A pre-PRM trial survey is proposed to understand decision-making at 
baseline. Survey and focus groups are proposed as a way to monitor implementation fidelity, 
including whether the PRM changes how decisions are made. The design calls for both formative and 
summative reports; the former are designed to shape implementation. 


Please check each of the following before submitting your Proposal: 

0 My firm is registered with the Allegheny County Department of Minority, Women and Disadvantaged 
Business Enterprises 

0 By submitting this proposal, I agree that, if offered a contract award, I will comply with the standard 
County terms and conditions for County contracts. 

0 By submitting this proposal, I agree to comply with DHS Cyber Security, EEOC/Non-Discrimination and 
HIPAA requirements. 

0 By submitting this proposal, I certify and represent to the County that all submitted materials are true 
and accurate, and that I have not offered, conferred or agreed to confer any pecuniary benefit or other 
thing of value for the receipt of special treatment, advantaged information, recipient's decision, opinion, 
recommendation, vote or any other exercise of discretion concerning this RFP. 
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COUNTY OF ALLEGHENY 
M/W/DBE PARTICIPATION WAIVER REQUEST 



W-9 


(Rev. December 2014) 
Department of the Treasury 
Interna! Revenue Service 


Request for Taxpayer 
Identification Number and Certification 


1 Name (as shown on your income tax return). Name is required on this line; do not leave this line blank. 

Chapin Hall Center for Children 

2 Business name/disregarded entity name, if different from above 


Give Form to the 
requester. Do not 
send to the IRS. 


3 Check appropriate box for federal tax classification; check only one of the following seven boxes: 
l~1 Individual/sole proprietor or [71 C Corporation □ S Corporation Q Partnership Q Trust/estate 
single-member LLC 

□ Limited liability company. Enter the tax classification (C=C corporation, S=S corporation, P=partnership) ► 

Note. Fora single-member LLC that Is disregarded, do not check LLC; check the appropriate box in the line above for 
the tax classification of the single-member owner. 

[7] Other (see instructions) ► 501 (c)(3) 


“ [7] Other (see instructions) ► _ 

£ 5 Address (number, street, and apt. or suite no.) 

g. 1313 East 60th Street _ 

^ 6 City, state, and ZIP code 

« Chicago, Illinois 60637 _ 

7 List account numbers) here (optional) 


Requester’s name 


4 Exemptions (codes apply only to 
certain entities, not individuals; see 
instructions on page 3): 

Exempt payee code (if any)_ 

Exemption from FATCA reporting 
code (it any) _ 

(Applies to accounts maintained outside the U.S.) 


and address (optional) 


Part I 


Taxpayer Identification Number (TIN) 


Enter your TIN in the appropriate box. The TIN provided must match the name given on line 1 to avoid 
backup withholding. For individuals, this is generally your social security number (SSN). However, for a 
resident alien, sole proprietor, or disregarded entity, see the Part I instructions on page 3. For other 
entities, it is your employer identification number (EIN). It you do not have a number, see How to get a 
TIN on page 3. 

Note. If the account is in more than one name, see the instructions for line 1 and the chart on page 4 for 
guidelines on whose number to enter. 


Social security number 




Employer identification number 


Certification _ 


Under penalties of perjury, I certify that: 

1. The number shown on this form is my correct taxpayer identification number (or I am waiting for a number to be issued to me); and 

2. I am not subject to backup withholding because: (a) I am exempt from backup withholding, or (b) I have not been notified by the Internal Revenue 
Service (IRS) that I am subject to backup withholding as a result of a failure to report all interest or dividends, or (c) the IRS has notified me that I am 
no longer subject to backup withholding; and 

3. I am a U.S. citizen or other U.S. person (defined below); and 

4. The FATCA code(s) entered on this form (if any) indicating that I am exempt from FATCA reporting is correct. 

Certification instructions. You must cross out item 2 above if you have been notified by the IRS that you are currently subject to backup withholding 
because you have failed to report all interest and dividends on your tax return. For real estate transactions, item 2 does not apply. For mortgage 
interest paid, acquisition or abandgnment of secured property, cancellation of debt, contributions to an individual retirement arrangement (IRA), and 
generally, payments other than intetest and dividends, you are not required to sign the certification, but you must provide your correct TIN. See the 
instructions on page 3. I / _ 


.► hs 


General Instructions 

Section references are to the Internal Revenue Code unless otherwise noted. 
Future developments. Information about developments affecting Form W-9 (such 
as legislation enacted after we release it) is at www.irs.gov/fw9. 

Purpose of Form 

An individual or entity (Form W-9 requester) who is required to file an information 
return with the IRS must obtain your correct taxpayer identification number (TIN) 
which may be your social security number (SSN), individual taxpayer identification 
number (ITIN), adoption taxpayer identification number (ATIN), or employer 
identification number (EIN), to report on an information return the amount paid to 
you, or other amount reportable on an information return. Examples of information 
returns include, but are not limited to, the following: 

• Form 1099-INT (interest earned or paid) 

• Form 1099-DIV (dividends, including those from stocks or mutual funds) 

• Form 1099-M ISC (various types of income, prizes, awards, or gross proceeds) 

• Form 1099-B (stock or mutual fund sales and certain other transactions by 
brokers) 

• Form 1099-S (proceeds from real estate transactions) 

• Form 1099-K (merchant card and third party network transactions) 


• Form 1098 (home mortgage interest), 1098-E (student loan interest), 1098-T 
(tuition) 

• Form 1099-C (canceled debt) 

• Form 1099-A (acquisition or abandonment of secured property) 

Use Form W-9 only it you are a U.S. person (including a resident alien), to 
provide your correct TIN. 

If you do not return Form W-9 to the requester with a TIN, you might be subject 
to backup withholding. See What is backup withholding? on page 2. 

By signing the fiiled-out form, you: 

1. Certify that the TIN you are giving is correct (or you are waiting for a number 
to be issued), 

2. Certify that you are not subject to backup withholding, or 

3. Claim exemption from backup withholding if you are a U.S. exempt payee. If 
applicable, you are also certifying that as a U.S. person, your allocable share of 
any partnership income from a U.S. trade or business is not subject to the 
withholding tax on foreign partners' share of effectively connected income, and 

4. Certify that FATCA code(s) entered on this form (if any) indicating that you are 
exempt from the FATCA reporting, is correct. See What is FATCA reporting? on 
page 2 for further information. 


Cat. No. 10231X 


Form W-9 (Rev. 12-2014) 


















6.3 Organizational Chart 


Organizational Chart 
Predictive Risk Model Evaluation 



28 

















* 

Kempe 

THE KEMPE CENTER FOR THE PREVENTION 
AND TREATMENT OF CHILD ABUSE AND NEGLECT 


December 17, 2015 

Fred Wulczyn, PhD 
Chapin Hall 


RE: Evaluation of a Predictive Risk Modeling Tool for Improving the Decisions of Child 
Welfare Workers 

Dear Dr. Wulczyn: 

The Kempe Center for the Prevention of Child Abuse and Neglect, Department of Pediatrics, University of Colorado 
School of Medicine is pleased to join Chapin Hall in submitting a response to the above referenced solicitation. 

The Kempe Center improves the care and well-being of children by strengthening families, communities, and the systems 
that serve them. The Kempe Center’s faculty are actively involved in research and evaluation that helps to strengthen the 
effective delivery of child welfare services and outcomes for children and families. The Kempe center is currently engaged 
with several evaluations in other states and counties including examination of assessment implementation, prevention 
programs, and family engagement enabling practices. 

For the proposed effort Kempe faculty, led by Dr. John D. Fluke, will work with you and Allegheny County to develop the 
evaluation design, coordinate data collection and analyze survey data. We are pleased to be able to contribute to the 
analysis of PRM model performance including work with administrative data, and be involved in the utilization and 
dissemination of results and findings. 

Please consider this letter as a commitment by the Kempe Center to work Chapin Hall should you be the successful 
bidder. We look forward to working with you and are confident that together we will contribute to the successful 
implementation of this important project. 


Sincerely, 



Desmond Runyan,™D, DrPH 

Jack and Viki Thompson Professor of Pediatrics, University of Colorado School of Medicine 
Executive Director, The Kempe Center | Children's Hospital Colorado 
Phone: 303.864.5333 | Fax: 303.864.5367 
Desmond. Runvan@childrenscolorado.orq 


The Gary Pavilion at The Children's Hospital - Anschutz Medical Campus 

13123 E. 16 th Avenue, B390 - Aurora, CO 80045 - p 303.864.5250 - www.thekempecenter.org 


University of Colorado 
SCHOOL OF MEDICINE 

Department of Pediatrics , / 
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Joyce James Consulting 

Equal Treatment Does Not Lead To Equity 


Director, Center for State Child Welfare Data 
Chapin Hall at the University of Chicago 
1313 East 60th Street 
Chicago, Illinois 60637 

Dear Dr. Wulczyn: 

On behalf of Joyce James Consulting, LLC (JJC), I am pleased to confirm our partnership with the Center for 
State Child Data at Chapin Hall (Chapin Hall) in connection with the Allegheny County Department of Human 
Services (DHS) solicitation requesting a process and impact evaluation of its child welfare Predictive Risk 
Model (PRM). 

JJC is a minority-owned limited partnership corporation, currently seeking women/minority owned business 
certification. Through its professional team of consultants, JJC has more than 35 years of experience in 
system transformation, research and development, organizational redesign, racial equity training, and 
strategic planning. Ms. James has thirty plus years' experience in child protective services (CPS) and public 
agency management, most recently as the Associate Deputy Executive Commissioner of the Center for 
Elimination of Disproportionality and Disparities and the State Office of Minority Health at the Texas Health 
and Human Services Commission. In this capacity, she provided the leadership, as directed by the Texas 
Legislature, for identifying and addressing disproportionality in child welfare, education, juvenile justice, and 
health. She served as the Assistant Commissioner of CPS for the Texas Department of Family and Protective 
Services between 2004 and 2009 where she spearheaded numerous nationally known research projects 
involving Risk Assessment, Family Group Decision Making, Kinship Care, Decision-Making and 
Disproportionality. Working within systems, JJC provides technical assistance and consulting, community 
organizing, leadership development, and policy analysis related to strategies to address disproportionality 
and racial inequities. 

Pursuant to its working partnership with Chapin Hall to design and implement a process evaluation and an 
impact evaluation of the PRM Tool, JJC will work in consultation with Allegheny County CYF and AUT 
Enterprises LTD to conduct focus groups and surveys to answer key questions found in the RFP. 

JJC shares with the Center for State Child Welfare Data at Chapin Hall the goal improving outcomes for all 
children in care. We are excited about this partnership and the opportunity to collaborate with Allegheny 
County DHS. 

Sincerely, 

Joyce James, LMSW-AP 
Joyce James Consulting, LLC 



2401 Silent Brook Trail • Round Rock, Texas 78665 • 409-553-0314, 512-244-1913 • joyce james 

consulting.com • jjameslmsw@aol.com 
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6.4 CVs/resumes 


Please see the following pages for the CVs/Resumes for all personnel associated with this project. 
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Curriculum Vitae 
Fred H. Wulczyn 

Office Address Chapin Hall at the University of Chicago 

1313 E. 60 th St. 

Chicago, Illinois 60637 
773-256-5200 
fwulczyn@uchicago. edu 

Education 

Ph.D. University of Chicago, 1986 

School of Social Service Administration 

M.S.W. Marywood University, 1979 

Graduate School of Social Work 

B.S. Juniata College, 1975 

(Psychology/Sociology) 

Awards 

2014 Fellow, American Academy of Social Work and Social Welfare 

2012 Doctor of Humane Letters, honoris causa, Marywood University 

2011 James E. Fly nn Prize for Research - University of Southern California School of 

Social Work 

2006 Peter A. Forsythe Award for Leadership in Child Welfare - National Association of 

Public Child Welfare Administrators 

2005 Distinguished Alumni Achievement Award - Juniata College 

2004 Distinguished Alumni Award - Marywood University 


Professional Employment 


2014-Present 

2010-2012 


2004-Present 


2006-Present 


Professor (part time), School of Social Service Administration, 
University of Chicago 

Special Assistant, Office of the Commissioner, Administration for 
Children, Youth, and Families, DHHS 

Director, Center for State Child Welfare Data, Chapin Hall at the 
University of Chicago 

Research Associate (Professor), University of Chicago 


2002-2006 Research Associate (Associate Professor), University of Chicago 

1996-2002 Research Associate (Assistant Professor), University of Chicago 


1996-Present 


Senior Research Fellow, Chapin Hall Center for Children at the 
University of Chicago 


1993-1997 

1986-1996 

1979-1986 


Assistant Professor, Columbia University School of Social Work 
Senior Policy Analyst, N. Y. Department of Social Services 
Research Associate, University of Chicago 



Publications 


Books 


Haskins, Ron, Fred Wulczyn, and Mary Bruce Webb (editors). Child Protection: Using 
Research to Improve Policy and Practice. Brookings Institution Press. (2007). 

Wulczyn, Fred, Richard Barth, Brenda Jones Harden, John Landsverk, and Ying-Ying 
Yuan. Beyond Common Sense: Child Welfare, Child Well-being, and the Evidence for 
Policy Reform. New York: Aldine Transaction, Inc. (2005). 

Peer-Reviewed Articles 

Chamberlain, P., Feldman, S. W., Wulczyn, F., Saldana, L., & Forgatch, M. (2015). 
Implementation and Evaluation of Finked Parenting Models in a Large Urban Child Welfare 
System. Child Abuse & Neglect, in press, 1-13. 

Wulczyn, F., Alpert, L., Monahan-Price, K., Huhr, S., Palinkas, & Pinsoneault, L. (2015). 
Research Evidence Use in the Child Welfare System. Child Welfare, In press, 1-17. 

Goldhaber-Fiebert, J. D., Babiarz, K. S., Garfield, R. L., Wulczyn, F., Landsverk, J., & Horwitz, 

S. M. (2014). Explaining variations in state foster care maintenance rates and the implications for 
implementing new evidence-based programs. Children and Youth Services Review, 39, 183-206. 

Holmes, L., Landsverk, J., Ward, H., Rolls-Reutz, J., Saldana, L., Wulczyn, F., & Chamberlain, 

P. (2014). Cost calculator methods for estimating casework time in child welfare services: A 
promising approach for use in implementation of evidence-based practices and other service 
innovations. Children and Youth Services Review, 39, 169-176. 

Wulczyn, F., & Landsverk, J. (2014). Research to practice in child welfare systems: Moving 
forward with implementation research. Children and Youth Services Review, 39, 145-146. 

Landsverk, J. A., & Wulczyn, F. H. (2013). Child Placement as a Response to Child Abuse and 
Neglect. Institute of Medicine. 

Wulczyn, F., Gibbons, R., Snowden, L., & Lery, B. (2013). Poverty, Social Disadvantage, and the 
Black/White Placement Gap. Children and Youth Services Review. 35, 65-74. 

Goldhaber-Fiebert, J. D., Snowden, L. R., Wulczyn, F., Landsverk, J., & Horwitz, S. M. (2011). 
Economic evaluation research in the context of Child Welfare policy: A structured literature 
review and recommendations. Child Abuse & Neglect, 35(9), 722-740. 

Goldhaber-Fiebert, J. D., Bailey, S. L., Hurlburt, M. S., Zhang, J., Snowden, L. R., 

Wulczyn, F., et al. (2011). Evaluating Child Welfare Policies with Decision-Analytic 
Simulation Models. Administration and Policy in Mental Health and Mental Health 
Services Research, 1-12. 

Wulczyn, Fred, Lijun Chen, and Mark Courtney. Permanency in Social Structural 
Context. Children and Youth Services Review. 2010. 

Wulczyn, Fred, Lijun Chen, and Britany Orlebeke. Evaluating Contract Agency 
Performance. Children and Youth Services Review. 2009. 

Wulczyn, Fred. Epidemiological Perspectives on Child Maltreatment. The Future of 
Children. 2009. 
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Wulczyn, Fred, Cheryl Smithgall, and Lijun Chen. Child Well-being: The Intersection 
of Schools and Child Welfare. Review of Research in Education. 2009. 

Wulczyn, Fred and Lijun Chen. Adoption Dynamics: Impact of the Adoption and Safe 
Families Act. Social Service Review. 2006. 

Wulczyn, Fred and Emily Zimmerman. Siblings in Foster Care: A Longitudinal 
Perspective. Children and Youth Services Review. 2005. 

Barth, Richard, Fred Wulczyn, and Tom Crea. From Anticipation to Evidence: Research 
on the Adoption and Safe Families Act. Virginia Journal of Social Policy and the Law. 
(2004/2005). 

Courtney, Mark, Barbara Needell, and Fred Wulczyn. Unintended Consequences of the 
Push for Accountability: The Case Of National Child Welfare Performance Standards. 
Children and Youth Services Review. 2004. 

Wulczyn, Fred. Reunification from Foster Care. The Future of Children, The David and 
Lucille Packard Foundation: 2004. 

Wulczyn, F., Joseph Kogan, and Brenda Jones Harden. (2002). Placement Stability and 
Movement Trajectories. Social Service Review, 2003 

Wulczyn, Fred. Closing the Gap: Are Changing Exit Patterns Affecting the Time African 
American Children Spend in Foster Care Relative to Caucasian Children. Children and 
Youth Services Review, 2003. 

Baker, Amy, Fred Wulczyn, and Nan Dale. Covariates of Length of Stay in Residential 
Treatment. Child Welfare. 2003. 

Wulczyn, Fred, Kristen Brunner, and Brenda Jones Harden, Infant Placements in Foster 
Care, Infant Mental Health Journal, June 2002. 

Wulczyn, Fred, Joseph Kogan, and John Dilts, The Effects of Population Dynamics on 
Performance Measurement, Social Service Review, June 2001. 

Wulczyn, Fred and Britany Orlebeke, Fiscal Reform and Managed Care in Child 
Welfare: Four Case Studies, Policy and Practice, September 2000. 

Wulczyn, Fred and Britany Orlebeke, Measuring Contract Agency Performance with 
Administrative Data, Child Welfare, June/July 2000. 

Wulczyn, Fred, Fiscal Reform in Child Welfare Services, Children and Youth Services 
Review, 2000. 

Goerge, R., and Wulczyn, F. Placement Experiences Of The Youngest Foster Care 
Population: Findings From The Multistate Foster Care Data Archive. Zero to Three, 

1998. 

Wulczyn, Fred, A Statistical and Methodological Framework for Analyzing the 
Experiences of Foster Children, Social Service Review, June, 1996. 

Goerge, Robert, Fred Wulczyn, and Allen Harden, New Comparative Insights into States 
and Their Foster Children, Public Welfare, Summer 1996. 
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Wulczyn, Fred, Robert Goerge, and David Fanshel, A Child Welfare Research Agenda, 

Child Welfare, Vol. LXXIII, No. 5, September/October, 1994. 

Wulczyn, Fred and Robert Goerge, Foster Care in New York and Illinois: The Challenge 
of Rapid Change, Social Service Review, 1992. 

Wulczyn, Fred. Caseload Dynamics and Foster Care Reentry. Social Service Review, 

March, 1991. 

Book Chapters 

Wulczyn, F., Bridgette Lery, & Lonnie Snowden. (2014). Poverty and the Black/White 
Placement Gap: Using Context to Understand the Role of Evidence-Based Interventions. In A. 
Shlonksy & R. Benbenishty (Eds.), From Evidence to Outcomes in Child Welfare: An 
International Reader. Oxford University Press. 

Wulczyn, F., Feldman, S., Alpert, L., & Horwitz, S. M. (2013). Maltreatment Prevention and the 
Science of Resource Allocation. In J. E. Korbin & R. Krugman (Eds.), Handbook of Child 
Maltreatment. Springer. 

Goerge, Robert, Fred Wulczyn, and Allen Harden, Foster Care Dynamics, in The Foster Care 
Crisis: Translating Research into Policy and Practice, Patrick Curtis, Grady Dale, and Joshua 
Kendall (eds.), University of Nebraska Press, 1999. 

Wulczyn, Fred, Child Welfare Outcomes: Some Methodological Considerations, in Outcomes 
Measurement, ed. by Edward Mullen and Jennifer Magnabosco. Washington, D.C.: NASW 
Press, 1997. 

Wulczyn, Fred, Child Welfare Reform, Managed Care, and Reinvestment in Urban 
Communities, in Children and Their Families in Big Cities, Sheila Kamerman and Alfred 
Kahn (eds.), Columbia University Press, New York, 1996. 

Wulczyn, Fred, HomeRebuilders: A Family Reunification Demonstration, in Barth, Rick, 

Jill Duerr-Berrick, and Neil Gilbert, Child Welfare Research Review, vol. II (New York: 
Columbia University Press, 1996. 

Wulczyn, Fred. Birth Status and Infant Foster Care Placements in New York City, in 
Barth, Rick, Jill Duerr-Berrick, and Neil Gilbert, Child Welfare Research Review, vol. I 
(New York: Columbia University Press, 1994). 

Reports and Other Publications 

Bartholet, Elizabeth, Fred Wulczyn, Richard P. Barth, and Cindy Lederman. (2011). 

Race and Child Welfare. Chicago: Chapin Hall at the University of Chicago. 

Wulczyn, Fred, Michelle Ernst, and Philip Fisher. (2011). Who Are the Infants in 
Out-of-Home Care? Chicago: Chapin Hall at the University of Chicago. 

Wulczyn, Fred, and Linda Collins. (2010). A 5-Year Projection in the Number of 
Children Turning 18 while in Foster Care. Chicago: Chapin Hall at the University of 
Chicago. 

Wulczyn, F. H., Daro, D., Fluke, J., Feldman, S., Glodek, C., & Lifanda, K. (2010). 

Adapting a Systems Approach to Child Protection. UNICEF. 
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Wulczyn, Fred, Britany Orlebeke, and Jennifer Haight. (2009). Finding the Return on 
Investment: A Framework for Monitoring Local Child Welfare Agencies. Chicago: 
Chapin Hall at the University of Chicago. 

Wulczyn, Fred, Lijun Chen, and Kristen Hislop. (2008). Foster Care Dynamics: An 
Update from the Foster Care Data Archive. Chicago: Chapin Hall Center for Children. 

Wulczyn, Fred. (2007). A Community’s Concern: Monitoring Outcomes in the Child 
Welfare System. New York City: Child Welfare Watch. 

Wulczyn, Fred and Bridgette Lery. (2007). Racial Disparities in Foster Care 
Admissions. Chicago: Chapin Hall Center for Children. 

Wulczyn, Fred. (2006). Monitoring Child Welfare Programs: Performance 
Improvement in a CQI Context. Chicago: Chapin Hall Center for Children. 

Wulczyn, Fred, Bridgette Lery, and Jennifer Haight. (2006). Exit and Entry Disparities 
in the Tennessee Foster Care System. Chicago: Chapin Hall Center for Children. 

Wulczyn, Fred and Britany Orlebeke. (2006). Getting What We Pay For: Do 
Expenditures Align with Outcomes in the Child Welfare System? Chicago: Chapin Hall 
Center for Children. 

Wulczyn, Fred. Child and Family Services: Dynamic Contradictions in the Search for 
Outcomes. Chicago: Chapin Hall Center for Children. 2003. 

Wulczyn, F., Emily Zimmerman, and Ada Skyles. (2002). Relative Care, Kinship Foster 
Care, and Subsidized Guardianship. Albany: New York State Office of Children and 
Family Services. 

Wulczyn, F. (2002). Adoption Dynamics: The Impact of the Adoption and Safe 
Families Act. Issue Papers in Foster Care and Adoption. Washington, D.C.: Department 
of Health and Human Services, Assistant Secretary for Planning and Evaluation. 

Wulczyn, F. and Hislop, K. B. (2002). Growth in the Adoption Population. Issue Papers 
in Foster Care and Adoption. Washington, D.C.: Department of Health and Human 
Services, Assistant Secretary for Planning and Evaluation. 

Wulczyn, F., Orlebeke, B., Haight, J., & Hislop, K. B. (2002). Differential Effects of 
ASFA on the Likelihood of Adoption. Issue Papers in Foster Care and Adoption. 
Washington, D.C.: Department of Health and Human Services, Assistant Secretary for 
Planning and Evaluation. 

Wulczyn, Fred, and Kristen Brunner, Children Under Age Three in Foster Care, 

Protecting Children, 2000. 

Wulczyn, Fred, Britany Orlebeke, Jennifer Haight, and Goerge Poede. The Child Welfare 
and Protection Project in Romania: Mid-term Evaluation. Chicago: Chapin Hall Center 
for Children, 2000. 

Wulczyn, Fred. Adoption Dynamics: A Report from the Multistate Foster Care Data 
Archive. Prepared for the David and Lucille Packard Foundation, Los Altos, Ca. June 
2000 . 
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Wulczyn, Fred, Emily Zimmerman, and Jennifer Haight. An Assessment of Contract 
Agency Capacity in the Washington, D.C. Child Welfare System. Chicago: Chapin Hall 
Center for Children, 2000. 

Wulczyn, Fred, Poverty, Cash Assistance, and Child Protection: Selected Trends From 
1990-1998, Prepared for the U.S. House Committee on Ways and Means. Chicago: 
Chapin Hall Center for Children, 2000. 

Harden, Allen, Fred Wulczyn, and Robert Goerge, Adoption from Foster Care: The 
Dynamics of the ASF A Foster Care Population (A Report from the Multi-state Foster 
Care Data Archive), Chicago: Chapin Hall Center for Children, 1999. 

Wulczyn, Fred, Robert Goerge, and Kristen Brunner, Foster Care Dynamics: An Eleven 
State Report from the Multistate Foster Care Archive, Chicago: Chapin Hall Center for 
Children, 1999. 

Wulczyn, Fred and Britany Orlebeke, Fiscal Reform in Child Welfare Services: Four 
Case Studies, Chapin Hall Center for Children, 1998. 

Wulczyn, Fred and Eileen Scheu, Setting Capitated Rates for Foster Care Programs, 
Chicago: Chapin Hall Center for Children, 1998. 

Wulczyn, Fred, Federal Fiscal Reform in Child Welfare Services, Chicago: Chapin Hall 
Center for Children, 1998. 

Wulczyn, Fred, In the Perfect World: Defining the State Of The Art in Outcome 
Measurement, in Sixth National Roundtable on Outcome Measures in Child Welfare 
Services: Summary of Proceedings, American Humane Association, Nancy McDaniel 
and Robyn Alsop, Eds., 1998. 

Wulczyn, Fred, Robert Goerge, and Allen Harden, Foster Care Dynamics: An Update 
from the Multistate Foster Care Archive, Chicago: Chapin Hall Center for Children, 

1997. 

Wulczyn, Fred, Foster Care Outcomes and Performance Standards: A Report to the Task 
Force on Performance Standards, New York City: Office of the New York City 
Comptroller, 1997. 

Wulczyn, Fred, Kinship Foster Care in New York Citv, (New York: Columbia University, 
1995) 

Wulczyn, Fred, Robert Goerge, and Allen Harden, Foster Care Dynamics in Five States: 
California, Illinois, Michigan, New York and Texas, Chicago: Chapin Hall Center for 
Children, 1993. 

Wulczyn, Fred. The Community Dimension of Permanency Planning, Albany: New York 
State Department of Social Services, June, 1991. 

Wulczyn, Fred and Robert Goerge, Public Policy and the Dynamics of Foster Care: A 
Multistate Study of Placement Histories, Chicago: The Chapin Hall Center for Children 
at the University of Chicago, 1990. 

Wulczyn, Fred, David Andersen, George Richardson, and Eric Wuestman. A System 
Dynamics Simulation of Caseload and Fiscal Implications of the Foster Care Baby Boom 
in New York City. (New York State Department of Social Services and the Rockefeller 
College of Public Affairs and Policy, 1990). 
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Wulczyn, Fred. The Distribution of Infant Foster Children in New York City. (New 
York State Department of Social Services, 1990). 


Wulczyn, Fred, David Andersen, George Richardson, and Eric Wuestman. Caseload and 
Fiscal Implications of the Foster Care Baby Boom. (New York State Department of 
Social Services, September, 1990). 


Wulczyn, Fred. The Changing Face of Foster Care in New York State. (New York State 
Department of Social Services, February, 1990). 


Wulczyn, Fred and Robert Goerge. Placement Duration and Foster Care Reentry in New 
York and Illinois: A Report to the U.S. Senate Finance Committee. (U.S. General 
Accounting Office, 1990. 


Wulczyn, Fred. The Dynamics of New York's Foster Care Caseload. (New York State 
Department of Social Services, March 1989). 


Bowen, Linda, Earl Johnson, Matthew Stagner, Fred Wulczyn, and Harold Richman. 
Adolescent Pregnancy and Public Policy: An Exploratory Study of the Relationship 
Between Adolescent Pregnancy and AFDC. Presented to the Joyce Foundation, Chicago, 
Illinois, 1985. 


Wulczyn, Fred. Understanding Caseload Dynamics. (New York State Department of 
Social Services, April 1987). 


Wulczyn, Fred. The New York State Child Support Guidelines. (New York State 
Department of Social Services, 1987.) 


Wulczyn, Fred. Questions and Answers About New York's Child Assistance Program. 
(New York State Department of Social Services, 1987). 


Wulczyn, Fred, Bob Goerge, Mary Ann Hartnett, and Mark Testa. Children in Substitute 
Care, in Mark Testa and Eddie Lawlor, The State of the Child: 1985. (The Chapin Hall 
Center for Children at the University of Chicago: 1986). 


John Schuerman, Fred Wulczyn, and Naomi Farber. The Evaluation of the Illinois 
Department of Public Aid's Young Parent Program: Interim Report. (Social Policy 
Research Center, NORC: 1984). 


Wulczyn, Fred. An Analysis of Children's Careers in Publicly Supported Child Welfare: 
1976-1979. (Children's Policy Research Project, University of Chicago, 1983). 


Wulczyn, Fred. The State of the Child. (Children's Policy Research Project, University 
of Chicago, 1980.) (With Mark Testa.) 


Selected Professional Activities 


Selection Committee. Innovations in Government Award. Harvard University, Kennedy 
School of Government. 2005. 
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Co-convener. Child Welfare Research Using the National Study of Child and Adolescent 
Well-being, in collaboration with Brookings Institution, 2005. 


Witness, Hearing on Federal Fiscal Reform, Testimony presented to the U.S. House of 
Representatives, Committee on Ways and Means, Subcommittee on Human Resources, 
June 2005. 


Co-convener. Strengthening Data Use and Analysis in Class Action Litigation, in 
collaboration with Children’s Rights, Inc., 2005 


Consultant. Pew Commission on Foster Care. 2004. 


Consultant. Implementation of Assembly Bill 636. California Department of Social 
Services, 2004. 

Series Editor, Chapin Hall Series on Child and Family Policy, in collaboration with 
Aldine DeGruyter, Inc. 


Special Issue Editor, Child Welfare Journal Issue on The Evidence Base for Child 
Welfare, September 2003. 


Witness, Hearing on Improved Monitoring of Vulnerable Children, Testimony presented 
to the U.S. House of Representatives, Committee on Ways and Means, Subcommittee on 
Human Resources, November 2003. 


Invited Speaker, Roundtable Discussion on Block Grants, The Brookings Institution, 
Washington, D.C., October 2003. 


Plenary Speaker, The Evidence Base for Child Welfare Policy, Child Welfare League of 
America Research Meetings, June 2001. 


Witness, Flexible Financing for Child Welfare, Testimony presented to the U.S. House of 
Representatives, Committee on Ways and Means, Subcommittee on Human Resources, 
2000 . 

Member, Child Welfare Research Advisory Panel, The New York Community Trust, 
2000 - 2001 . 


Convener, The Evidence Base for Child Welfare Policy and Practice, Chapin Hall Center 
for Children at the University of Chicago, December, 2000. 


Presenter/Discussant, Panel on Flexible Funding for Federal Child Protection Programs, 
U.S. House Committee on Ways and Means and the American Public Human Services 
Association, May, 2000. 


Member, National Advisory Group, National Resource Center on Child Welfare and 
Information Technology. 
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Presenter/Discussant, Welfare Reform and Child Protection, U.S. House Committee on 
Ways and Means, Subcommittee on Human Resources, Lecture Series, March, 2000 


Convener, Working Conference on Federal Financing for Child Welfare Sendees, U.S. 
Capitol, 1999. 


Consultant, Child Care Panel, National Academy of Sciences, 1999-2000. 


Consultant, Task Force on Child Welfare Finance Reform, American Public Human 
Services Association, 1998-2000. 


Discussant, Child Care Outcome Measurement, National Academy of Sciences, 1999. 

Discussant, Panel on Kinship Foster Care, APPAM, October, 1999. 

Expert witness, Marisol v. Guiliani, New York City Department of Law, 1998. 

Member, Board of Directors, The Child Welfare Watch, Center for an Urban Future, 
New York City, 1995-2006. 

Co-chair, New York City Administration for Children’s Services, Advisory Panel on 
Fiscal Reform, 1996-2000. 


Member, New York City Administration For Children’s Services, Commissioners ’ 
Advisory Panel, 1996-2006. 


Member, New York City Comptroller’s Advisory Panel on Foster Care Performance and 
Outcome Measures, 1995-1998. 


Member, Bronx Borough President’s Advisory Panel on Child Welfare Sendees, 1995- 
1999. 

Member, Technical Advisory Panel, National Evaluation of Family Preservation and 
Support Programs, U.S. Department of Health and Human Services, 1994-2000. 


Member, Technical Advisory Panel, National Study of Kinship Foster Care, U.S. 
Department of Health and Human Services, 1993-1996. 


Selected Other Professional Papers 


Testing the Racial Invariance Hypothesis: The Case of Racial Disparities in Foster Care 
Admission Disparities. Society for Social Work Research Annual Conference, 2009. 


Transition to School for Infants in the Child Welfare System. Association for Public 
Policy and Management Annual Research Meeting, 2008. 
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Racial Disparities in Foster Care Admissions. Association for Public Policy and 
Management Annual Research Meeting, 2008. 


Substance Abuse and the Basic Epidemiology of Child Maltreatment and Foster Care 
Placement. Invited Paper. National Institute of Drug Abuse. 2005 


Place Matters: How are Foster Care Placement Rates Related to Poverty ? Childhoods 
2005. Oslo, Norway. 2005. 


Events and Trajectories: Understanding Maltreatment Recurrence and Foster Care 
Placement. 15 th National Conference on Child Abuse and Neglect. Boston, Ma. 2005 


Poverty and Placement in Foster Care. 15 th National Conference on Child Abuse and 
Neglect. Boston, Ma. 2005 


Outcome Measurement in Child Welfare. University of New South Wales, Sydney, 
Australia, 2004. 


Infant Placements in Foster Care, Permanent Judicial Commission on the Rights of 
Children, 2000. 


A Comparative Analysis of Urban Child Welfare Systems, Urban Child Welfare 
Leadership, Annie E. Casey Foundation, 2001. 


Sixth Annual Silverstein Memorial Lecture, Staten Island Mental Health Center, 
November, 1998. 

Fiscal Reform for Child Welfare Sendees, Plenary Speaker, National Association of 
Public Child Welfare Administrators, Washington, DC, 1997 


Child Welfare Outcomes: Integrating Research and Practice, Fourth Annual Roundtable 
on Child Welfare Outcomes, American Humane Association, San Antonio, 1997. 


Child Welfare Managed Care: A State Policy Perspective, Family Resource Coalition’s 
6th Annual National Conference, 1996 


Child Welfare Reform and Managed Care, National Association of Public Child Welfare 
Administrators, New Orleans, 1995. 


Evaluation and Program Monitoring: Implications for the Family Support and Family 
Preservation Legislation, U.S. Department of Health and Human Services Planning 
Conference, New York City, March 1994. 


The Multistate Foster Care Archive: Year 1 Results, National Child Welfare Conference, 
Washington, D.C., March, 1994. 


Recent Developments in the Study of Caseload Dynamics, The Family Welfare Research 
Group, University of California, Berkeley, August, 1992. 
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National Governor's Association, Expanding the Analytical Value of Administrative Data, 
January, 1992. 


Administrative Data Institute, Consultant, Chapin Hall Center for Children, September, 
1991. 


National Symposium on Child Welfare, Sponsored by the U.S. Department of Health and 
Human Services, New York State Department of Social Services, and the Chapin Hall 
Center for Children, May, 1991 (co-convener). 


Multistate Perspectives on Foster Care. APPAM Research Meeting, San Francisco, 
October, 1990. 


Foster Care Research in New York and Illinois. McMaster University Symposium on 
International Perspectives on Foster Care, April, 1990 


Caseload Dynamics and Foster Care. International Systems Dynamics Conference, 
Chestnut Hill, Massachusetts, July 1990. 


Evaluation in the Human Sendees. National Governor's Association, New Orleans, 
Louisiana, June, 1990. 


On the Fractal Qualities of Caseload Dynamics. National Association for Welfare 
Research and Statistics, Louisville, Kentucky, August, 1987. 


The Problem of Cross Level Inference. Evaluation Research Society meetings, Toronto, 
Canada, 1985. 


Summer Faculty, School of Social Service Administration, University of Chicago. 
Course: Program Development and Evaluation : Applied Survey Research. Summer, 
1985. 

Administrative Data: Polishing a Tarnished Image. Evaluation Research Society 
meetings, San Francisco, 1984. 


Building a Social Experiment from Survey Data: Maximizing Policy Impact. Evaluation 
Research Society meetings, San Francisco, 1984. 


Children and Youth: The Changing Fabric of American Society. Keynote address, 
Education and the Changing Fabric of American Society. A symposium sponsored by 
Phi Delta Kappa and the School of Education, Northwestern University, 1983. 


Funded Research 


Principal Investigator. Research Evidence Use in Private Child Welfare Agencies. W. T. Grant 
Foundation. $250,000. 
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Principal Investigator. Simulation Model of Long Run Demand for Residential Care. Center for 
State Foster Care and Adoption Data, $25,000. (with the Argonne National Laboratory, Center 
for Advanced Simulation). 


Principal Investigator, Comparative Analysis of Provider Performance. Casey Family Programs. 

$ 200 , 000 . 


Principal Investigator. Comparative Analysis of Placement Stability. Partners for Our Children, 
2008. $100,000. 

Principal Investigator. A Longitudinal Study of Children in Out-of-Home Care. Department of 
Community Services, New South Wales, Australia. 2008-2012. $1.7 million. 


Principal Investigator, Monitoring the Child Welfare System in South Florida. State of Florida. 
2007-2009. $1.5 million 


Principal Investigator, Center for State Foster Care and Adoption Data. Annie E. Casey 
Foundation and various states. $145,000. 


Principal Investigator, Evaluation of Tennessee Contract Provider Performance. State of 
Tennessee. 2003-2009. $1.2 million. 


Principal Investigator, Evaluation of the Partnership for Social Change - Highbridge. Fund for 
Social Justice. 2003-2006. $210,000 


Principal Investigator (in collaboration with New York City). An Infant System of Care in 
Bedford-Stuyvesant. The U.S. Department of Health and Human Services, 2003-2008. $2.5 
million 


Principal Investigator, The Child Welfare Research Project, The Rutgers University School of 
Social Work and the New Jersey Division for Youth and Family Services, 2002-2004. $580,000 


Principal Investigator, The Evidence Base for Child Welfare Policy, The David and Lucille 
Packard Foundation, 2001. $75,000. 


Principal Investigator, Research Support for NYC: Integrating Research, Policy, and Practice. 
Administration for Children’s Services. 1998-2007. $2.6 million. 


Principal Investigator, STARDAT: Web-based Decision Support Systems for Child Welfare, 
Annie E. Casey Foundation, 2000. $125,000 


Principal Investigator, Coping Strategies Among Low-Income Working Families, Assistant 
Secretary for Planning and Evaluation, U.S. Department of Health and Human Services, 2000- 
2001. $155,000 


Principal Investigator, The Multistate Foster Data Archive, Assistant Secretary for Planning and 
Evaluation, U.S. Department of Health and Human Services, and the Annie E. Casey Foundation, 
2001. $250,000 
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Principal Investigator, The Child Welfare and Protection Project in Romania, World Vision/U.S. 
Agency for International Development, 2000. $75,000 


Principal Investigator, Foster Care Utilization and Budget Impacts, New Jersey Division of Youth 
and Family Services, 2000. $60,000 


Principal Investigator, Youth Transitions in Independent Living, Casey Family Program, 1999- 

2000 . $ 20 , 000 . 


Principal Investigator, A Survey of Child Welfare Provider Capacity, Washington, D.C., the 
Annie E. Casey Foundation and the Office of the General Receiver, Washington, D.C., 1999. 
$ 68 , 000 . 


Principal Investigator, The Protect Ohio Title IV-E Project Evaluation, State of Ohio, 1998-2003. 
$360,000. 


Principal Investigator, The Utilization of Mental Health Services by Children, Integrated 
Children’s Services, 1998-1999. $100,000 


Principal Investigator, Evaluation of the Massachusetts Commonworks Program, Massachusetts 
Department of Social Services, 1998-2001. $60,000 


Principal Investigator, "Multistate Foster Care Data Archive." Department of Health and Human 
Services, Administration for Children Youth and Families (1998-2001). $750,000 


Principal Investigator, The Core Child Welfare Database: A Joint Project of the Administration 
for Children’s Services in New York City and the Chapin Hall Center for Children, Annie E. 
Casey Foundation, 1997. $250,000 

Principal Investigator, Child Welfare Managed Care: Evaluation Issues, Smith Richardson 
Foundation. $135,000. 


Principal Investigator, Developing New Fiscal Strategies in Child Welfare (Technical 
Assistance), The Annie E. Casey Foundation, 1996. $125,000. 


Principal Investigator, Child Welfare Managed Care Forum, The Foundation for Child 
Development (1996-1997). $50,000 


Principal Investigator, Child Welfare Managed Care Forum, The Edna McConnell Clark 
Foundation (1996-1997). $75,000. 


Principal Investigator, "Monitoring Child Welfare Outcomes." The Child Welfare Fund (1994- 
1995). $45,000. 


Principal Investigator, "A Study of Kinship Foster Care in New York City." The Robert Sterling 
Clark Foundation (1993-1994). $60,000. 
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Principal Investigator, "A Study of HomeRebuilder Fiscal Changes." The Child Welfare Fund 
(1993). $25,000. 


Principal Investigator, "The Impact of HomeRebuilders on Child Welfare Outcomes." New York 
Community Trust (1993-1994). $40,000 


Co-director, "Multistate Foster Care Data Archive." Department of Health and Human Services, 
Administration for Children Youth and Families (1992-1997). $750,000. 


Co-Principal Investigator, "Public Policy and the Dynamics of Foster Care." Department of 
Health and Human Services, Assistant Secretary of Planning and Evaluation (1989-1991), 
Foundation for Child Development, and New York Community Trust (1989-1990). $125,000. 

Co-Principal Investigator, "Duration and Foster Care Reentry in New York and Illinois." U.S. 
General Accounting Office (1989-1990). $25,000. 
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Lijun Chen 


Home Address: 



Tel: iHH 

Email: lchenl@uchicago.edu 
Education 


Office Address: 

Chapin Hall 

1313 East 60 th Street 

Chicago, IL 60637 

773-256-5140 

lchen@chapinhall.org 


Ph.D. University of Chicago, Sociology June 2004 
M.A. University of Chicago, Sociology July 1999 
M.A. Beijing Foreign Studies University, English Literature 1991 

Fellowships and Awards 


Dissertation Writing Fellowship, Center for East Asian Studies, University of Chicago, 2001-2002 
Young Scholars Award, China Times Foundation, New York, 2000-2001 
Midwest Sociological Society Scholarship Award, 1999-2000 

Work Experience 


Senior Researcher, Chapin Hall at the University of Chicago May 2009- Present 

Senior Research Specialist, Chapin Hall Center at the University of Chicago Aug 2007- May 2009 

Researcher, Chapin Hall Center at the University of Chicago Aug 2003- Aug 2007 

* Conducting policy research on children in the welfare system with advanced data modeling methods based 
on large-scale survey data and administrative data from states and child welfare agencies. 

* Working with child and youth service organizations to design, implement and conduct reviews and 
evaluations of their intervention programs. 

* Conducting multinational comparative research on development and wellbeing of children in other 
countries through collaboration with researchers in international organizations and foreign countries such as 
UNICEF and China. 


Research Assistant, University of Chicago various periods, 1996 - 2001 

* Performed data coding, cleaning and analysis for Gang Research, Evaluation and Technical Assistance (G 
RETA) projects directed by Professor Spergel (Jun 2000 - Oct 2000, Apr 2001- Sep 2001) 

* Analyzed the national survey data of Chinese private entrepreneurs, and looked into the government- 
business relationship through survey analysis under supervision of Professor Parish (Oct 1999 - June 2000) 

* Checked and cleaned FAUI (fiscal austerity and urban innovation) survey data, and prepared and checked 
syntax files for statistical analysis using SPSS under supervision of Professor Clark (Sep 1998 - June 1999) 

* Conducted statistical analysis commissioned by University of Chicago faculty and other clients on various 
topics such as teenage sexual behavior, sexually transmitted diseases and the Christian religion 

Teaching Assistant, University of Chicago various times, 2002 - 2003 

* Led the weekly review sessions, graded student assignments, and offered training sessions in SPSS for 
Statistical Methods of Research offered by Professor Dingxin Zhao, January 2003 - March 2003 

* Formulated exam questions, graded exams, led weekly discussions, and advised student theses for Political 
Sociology offered by Professor Terry Clark, March 2002 - May 2002 

Legal assistant, Paul, Weiss, Rifkind, Wharton & Garrison, Beijing, China Aug 1993 - May 1996 
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Drafted and proofread legal documents in English and Chinese; participated and interpreted during business 
negotiations between American and Chinese clients; conducted research on Chinese economic policies and 
compiled industrial sector profiles. 

Press Officer, The Chinese Ministry of Foreign Affairs, Beijing, China Apr 1991 - July 1993 

Interpreted and translated for press conferences and on other occasions; monitored, synthesized and 
translated foreign news agency releases; prepared daily newsletters and briefed ministry officials on 
international affairs. 

Part-time Lecturer, Beijing West District Adult Education College, Beijing, China Aug 1989 - Aug 1992 
Taught courses in English language and literature. 

Teacher, Gaotang No. 1 High School, Shandong, China Sep 1984 - Jul 1986 

Taught the English language to high school students, and organized “English comer” and other language 
improvement activities. 

Publications and Presentations 

Lijun Chen and Dali Yang. 2012. “Old Age Care Concerns and State-Society Relations in China: Public Anxiety and 
State Paternalism,” Journal of Asian Public Policy 5 ( 2): 136-154. 

Fred Wulczyn, Lijun Chen, Linda Collins, and Michell Ernest. 2011. “The Foster Care Baby Boom Revisited: what do 
the numbers tell us?” Zero to Three 31(3): 4-10. 

Fred Wulczyn, Lijun Chen, and Mark Courtney. 2011. “Family Reunification in a Social Structural Context.” 

Children and Youth Services Review 33 (3): 424-430. 

Fred Wulczyn, and Lijun Chen. 2010. “Placement Stability and Movement Trajectories.” In R. Barth and E. Fernandez 
(eds.) How Does Foster Care Work? International Evidence on Outcomes. London: Jessica Kingsley Publishers. 

Fred Wulczyn, Lijun Chen, and Orlebeke Britany. 2009. “Evaluating Contract Agency Performance in Achieving 
Reunification.” Children and Youth Services Review 31 (5): 506-512. 

Fred Wulczyn, Cheryl Smithgall, and Lijun Chen. 2009. “Child Well-Being: The Intersection of Schools and Child 
Welfare.” Review of Research in Education 33: 35-62. 

Fred Wulczyn, and Lijun Chen (November 8, 2008). “Transition to School: The Impact of Child Welfare Services on 
School Readiness.” 30 th Annual Association for Public Policy Analysis and Management (APPAM) Fall Research 
Conference, Los Angeles, CA. 

Fred Wulczyn, Lijun Chen, and Kristen Brunner Hislop. 2006. “Adoption Dynamics and the Adoption and Safe 
Families Act.” Social Service Review 80 (4): 585-608. 

Research Reports 

John Fluke, Fred Wulczyn, Katherine Casillas, Lijun Chen, Claudia Cappa. 2010. Child Disciplinary’ Practices at 
Home: Evidence from a range of low and middle-income countries. United Nations Children’s Fund (UNICEF), 
Division of Policy and Practice. New York. Available online at http://www.childinfo.org/files/report_Disipl_FIN.pdf. 

Fred Wulczyn, Lijun Chen, Kristen Brunner Hislop. 2007. Foster Care Dynamics 2000-2005: A Report from the 
Multistate Foster Care Data Archive. Chapin Hall at the University of Chicago, Chicago. Available online at 
www.chapinhall.org. 

Fred Wulczyn, Kristen Brunner Hislop, Lijun Chen. 2005. “Adoption Dynamics: An Update on the Impact of the 
Adoption and Safe Families Act.” Chapin Hall at the University of Chicago. Available online at www.chapinhall.org. 
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Training in Data Analysis and Statistical Methods 


Oct 5-6, 2012 
Aug 6-20, 2010 
June 22-24, 2009 
Oct 27-29, 2008 
Aug 20-22, 2007 
July 2005 


Workshop on Analysis of Missing Data - Chicago, IL 
Workshop on Research Design for Causal Inference - Chicago, IL 
Longitudinal Exploration of Child Outcomes and Well-being - Ann Arbor, MI 
Advanced Longitudinal Analysis Workshop Using Mplus - Denver, CO 
Multilevel Modeling Using Mplus - Baltimore, MD 
Spatial Data Analysis Using GeoDA - Urbana, IL 
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JIANYU WANG 


Skills 

Oracle Database Administration, performance tuning, backup, and network 
services 

Sybase, MySQL database experience 

SFTP, SSH, SSL, X-window 

Research design and research methodology 

From scratch website design and implementation 

Perl, PHP, PL/SQL, T-SQL, Shell scripts, HTML, C++, SAS, JSP 

GIS system, record matching, and geocoding applications 

Software development, technical architecture integration 

Unix system, Linux system, Windows system 

Staff supervision and user training 

External consultation and vendor management 

Experience 

Senior Database Administrator, Chapin Hall Center for Children: 2001-Present 
Data Administration and Processing 

► Designed large size Oracle data warehouse for administrative data for over 20 
states 

► Created, populated, and managed Sybase tables and indexes 

► Managed MySQL objects for WordPress and other web applications 

► Developed backup plan to ensure high database availability 

► Execute database performance tuning process at I/O levels, user SQL, and 
schema design 

► Wrote robust Perl, Shell scripts, T-SQL, Oracle PL/SQL programs for data 
processing, including data cleaning, analyzing, data loading, and merging. 

► Created schemas with different roles and privileges for data security and access 
control 

► Received data by SFTP, tapes, ACCESS dump, and Oracle dump format 

► Create ORTHO parameter file for processing binary or text files. 

► Automated complicated, multi-stage processing into one-click program to 
drastically improve performance and minimize user error 

Application development and product testing 

Designed a 3-tier Cloud Service with Apache as the middle tier and Oracle as the 
backend 


773-256-5172 

jwang@chapinhall.org 

Chapin Hall 
1313 E. 60th St. 
Chicago, IL 60637 
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Implemented on-the-fly execution of some statistics models based on user' s 
options 

Streamlined Automatch steps for record linking and geocoding 
Installed and tested new matching software Netrics 
Tested record matching programs such as BigMatch and Linkplus 
Wrote programs to dynamically generate online maps using freeware 

System experience 

On Unix/Solaris, installed and configured Oracle 8i, 9i database management 
system 

On Linus/SUSE, installed and configured Oracle lOg and llg 
Installed different versions of PHP 

Updated and patched WordPress, Oracle database on a regular basis 
Installed and configured Apache server on both Unix/Linux box 
Installed JPgraph, WordPress and many other software 

Network and data security 

Configured Oracle Net service for communication between database and web 
server 

Wrote Oracle listener.ora and tnsnames.ora for Linux users to access Oracle 
through network 

Configured Tomcat6 to create "connection pool" to significantly reduce 
response time 

Generated a self-signed SSL certificate and Private key for Apache server to 
reduce cost 

Found and Fixed several secure holes on web applications 
Other experience 

Supervised and trained staff in SAS, Linux, SQL, and data cleaning skill 
Evaluated and selected external vendors on a cost-effective basis 
Managed three external consulting companies for product development 
Communicated to organization, coworkers, and users about technology plans 
Served 8 years on Habitat Committee 

Junior Oracle DBA, Teltech Consulting, Lisle, Illinois: 2000 - 2001 
Responsibilities 
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Oracle 8i administration and Performance tuning 
PL/SQL programing development 

Consultant, Solo Sessions, Chicago, Illinois: 1999 - 2000 
Responsibilities 

• Health, Self-awareness, and fitness consulting 
Single user accounting database administration 


Education 

The Ohio State University, Columbus, Ohio - Ph.D., 1998 
Exercise Physiology (Minor in Statistics) 

Beijing University of P. E., Beijing, China - Master of Science, 1992 
Biology 

Shandong University, China - Bachelor of Science, 1989 
Biology and Mathematics 

Certification 

Oracle Certified Professional 11 g (OCP) 

Oracle SQL and PL/SQL (F31DTT160B) 

Oracle Database Administration (F14DTT1305) 

Oracle Performance Tuning (F1CDTT0A09) 

Oracle Backup and Recovery (F23DTT1128) 
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Dana M. Hollinshead, Ph.D. 


Skill Summary 

• Extensive child welfare project management, research, program evaluation, and policy analysis 
experience utilizing advanced quantitative and qualitative data collection, analysis, and writing 
skills. 

• Insight into federal, state, and local perspectives on child welfare programs, policies, practices, 
administrative data, performance measurement, program improvement, and system refonn efforts. 

• Expertise examining a comprehensive array of factors affecting child welfare outcomes. 

• Demonstrated presentation, training, technical assistance, publication, grant writing and grant 
reviewing experience. 

Employment History 

2013 - PresentAssistant Research Professor, Kempe Center for the Prevention and Treatment of Child 
Abuse and Neglect, University of Colorado School of Medicine, Denver, CO. 

2003 - 2008 Child and Family Services Review and Title IV-E Review Consultant Reviewer, 
Johnson, Bassin, & Shaw, Silver Spring, MD. 

2001 - 2002 Child Welfare Program Specialist, United States Department of Health and Human 
Services, Administration for Children and Families, Boston, MA. 

2000 - 2001 Senior Research, Program, and Policy Analyst, American Humane Association, 
Children’s Division, Englewood, CO. 

1997 - 2000 Program, Policy, and Research Analyst, American Humane Association, Children’s 
Division, Englewood, CO. 

1996 - 1997 Research and Policy Associate, Center for the Study of Social Policy, Washington, DC. 

1994 - 1996 Research Assistant Intern, Child Welfare League of America, Washington, DC. 

Education 

2012 PhD Social Policy, Children, Youth, and Families Policy concentration. Heller School of 
Social Policy and Management, Brandeis University, Waltham, MA. Dissertation : Do 
Outcomes Reflect Performance, Demographic Dividends, or Policy Artifacts? A Study of 
the Effects of Child Welfare Policies, Practices, and Demographics on State Child 
Maltreatment Recurrence Rates. 

1995 MPA Policy Analysis and Program Evaluation emphasis, Child and Family Studies 

Concentration. Trachtenburg School of Public Policy and Public Administration, George 
Washington University, Washington, D.C. 

1989 BA Psychology. Colby College, Waterville, ME. 

Select Child Welfare Research, Program Evaluation, and Policy Analysis Projects 
Kempe Center for the Prevention and Treatment of Child Abuse and Neglect Projects: 

1) Tennessee IV-E Waiver Evaluation. Senior Research Analyst, Developed child welfare staff survey 
and analyzed data examining caseworker, agency and case-level factors affecting effective 
implementation of new case practices targeted for an out-of-home placement population. 

2) Found Engaged and Connected, Olmsted County, MN Family Group Conference Evaluation. 
Senior Research Analyst. Analyzing data concerning program fidelity, staff perceptions, and 
participant outcomes for a propensity score matching study examining the influence of family group 
conferences on high risk child welfare, juvenile justice, and/or behavioral health services youth. 

3) New York Administration for Children’s Services Enhanced Family Conferencing Initiative. 

Senior Research Analyst. Developing formative, implementation, and outcome evaluation examining 



role of parent advocates in enhancing family engagement in child welfare services. 

4) No Place Like Home Family Group Decision Making Evaluation. Senior Research Analyst. 
Conducted decision-making ecology research in three sites implementing family group decision making 
to examine implementation, utilization, and comparative outcomes for in home services populations 
using randomized control trial methodology. 

5) Quality Improvement Center on Differential Response in Child Protective Services Cross-Site 
Evaluation. Senior Research Analyst. Conducted data analyses, wrote reports and articles, and 
developed and presented results for a project examining the processes and outcomes of a randomized 
control study of three sites’ differential response programs, with a particular emphasis on factors 
affecting family engagement, service utilization, and outcomes. 

6) Saudi Arabian Adverse Childhood Experiences (ACEs) Study. Conducted multivariate data analyses 
for a national study exploring association between ACES, demographics, and health or risky behavior 
outcomes. 

US DHHS Administration for Children and Families Program Reviews and Oversight Activities: 

1) Child and Family Services Reviews (CFSR) and Title IV-E Eligibility Reviews. Lead/Reviewer. 

2) Title IV-E, CFSR, Child Abuse Prevention and Treatment Act, SACWIS and AFCARS Program 
Improvement Plan and Annual Report Approvals. State-Specific or Regional Liaison. 

American Humane Association or Center for the Study of Social Policy Projects: 

1) Assessment of the District of Columbia Bifurcated Intake and Investigation System. Project 
Manager and Senior Research and Policy Analyst. 

2) Dynamics of Unsubstantiated Reports of Child Abuse and Neglect. Policy and Research Analyst. 

3) California DSS Workload Study. Research Analyst and Technical Assistant. 

4) Comprehensive Review of the Cuyahoga County, Ohio DCF. Program, Policy, Research Analyst. 

5) Iowa DCFS Outcome-Based Performance Monitoring System Project. Policy Analyst. 

6) Arkansas DCFS Case Record Review and Site Visits. Research Associate. 

7) Community-Based Child Protective Services Evaluation Workgroup. Research Associate. 

Select Publications 

1) Hollinshead, D., Kim, S., Fluke, J., & Merkel-Holguin, L. (2015). The influence of family, agency, 
and caseworker dynamics on caregivers’ satisfaction with their child protective services intervention. 
Journal of Public Child Welfare. DOI: http://dx.doi.org/10.1080/15548732.2Q15.1091762 

2) Williams, J., Merkel-Holguin, L., Allan, H., Maher, E., Fluke, J., & Hollinshead, D. (2015). Factors 
associated with staff perceptions of the effectiveness of Family Group Conferences. Journal of the 
Society for Social Work & Research, 6(3), 343-366. 

3) Merkel-Holguin, L., Hollinshead, D., Hahn, A., Casillas, K., & Fluke, J. (2015). The influence of 
differential response and other factors on parent perceptions of child protection involvement. Child 
Abuse and Neglect, 39, 18-31. 

4) Fluke, J., Shusterman, G., Hollinshead, D., & Yuan, Y.Y. (2008). Longitudinal Analysis of 
Repeated Child Abuse Reporting & Victimization: Multistate Analysis of Associated Factors, Child 
Maltreatment, 13, 76-88. 

5) Hollinshead, D., & Fluke, J. (2001). What Works in Safety and Risk Assessment for Child 
Protective Services. In M. Kluger, G. Alexander & P. Curtis (Eds.), What Works in Child Welfare. 
(67-92). Washington, DC: CWLA. 

6) American Humane Association (AHA). (1999). Developing and Implementing an Outcome-Based 
Performance Monitoring System for Child Welfare Services in Iowa: Final Report. Englewood, CO. 


Hollinshead p.2 




John D. Fluke 


RESUME 


PERSONAL HISTORY OR BIOGRAPHICAL SKETCH: 

Kempe Center for the Prevention and Treatment of Child Abuse and Neglect, Department of 
Pediatrics, School of Medicine University of Colorado, Anschutz Campus, Aurora, CO 

Associate Director, Systems Research and Evaluation and Research Associate Professor (2012 to present) 

The Gary Pavilion at Children’s Hospital Colorado | Anschutz Medical Campus 

13123 East 16th Avenue, B390 | Aurora, CO 80045|Phone: (303) 864-5219 | Fax: (303) 864-5367 

john.fluke@ucdenver.edu 

Dr. Fluke has over 35 years of experience in social service delivery system research in the area of Child 
Welfare and Mental Health Services for children. He is internationally recognized as a researcher 
specializing in assessing and analyzing decision making in human services delivery systems. He is also 
known for his innovative and informative research and evaluation work in the areas of child maltreatment 
prevalence, child welfare administrative data analysis, workload and costing, and performance and 
outcome measurement for children and family services. He is active in the area of national child 
maltreatment administrative and self-report data collection systems and analysis and has worked with data 
collection programs in the Balkans, Canada, Saudi Arabia, the US, and for UNICEF. He has conducted 
research and evaluation at all levels of government, in the private not-for-profit sector, and with national 
foundations and associations that includes work both in the U.S. and internationally. 

At the Kempe Center Dr. Fluke is pursuing ongoing research in the area of child welfare decision making, 
administrative data analysis, and the implementation and scale up of evidence supported interventions. 

EDUCATION: 

Ph.D., Organizational Decision Science, Union Graduate School, Cincinnati, Ohio, 1995. 

Periodic Graduate course work in Economics (Econometrics), Statistics and Mathematics at the University 
of Colorado in Boulder and Denver, 1981-1990. 

M.A., Anthropology, Pennsylvania State University, University Park, Pennsylvania, 1980. 

B.A., Summa Cum Laude, Individualized Tutorial Program (ITP), Emphasis in Mathematical 
Anthropology, University of Northern Colorado, Greeley, Colorado, 1976. 

ACADEMIC APPOINTMENTS: 

Graduate School of Social Work, University of Denver, Denver, CO 
Visiting Scholar (2008 to Present) 

In his role as visiting scholar at Graduate School of Social Work Dr. Fluke assists the school to improve 
its approach to developing its child welfare capacity through strategizing with faculty and staff and fulfills 
educational requirements to provide periodic lectures and other substantive content for faculty and 
students. 




Factor Inwentash Faculty of Social Work, University of Toronto, Toronto, Ontario, Canada 

Status Only Assistant Professor (2012 to present) 

This appointment facilitates coordinated collaboration in the area of research grants, projects and 
publications. This also includes the annual development and teaching of a graduate level course in child 
welfare decision making. 

HOSPITAL, GOVERNMENT OR OTHER PROFESSIONAL POSITIONS: 

The American Humane Association, Children's Division, Englewood, CO 
Director of Program Analysis and Research (1979-1999) 

Senior position involved primary responsibility for the analytic consultation, research, and evaluation 
performed by the division. Supervision of the agency's research staff consisting of eight research 
professionals. The work focused on general project design and implementation related to public child 
protective services and community-based program evaluation for state social service agencies across the 
country, for the Federal Government and other agencies providing children’s services. 

Walter R. McDonald & Associates, Inc,, Aurora, CO 
Director of Research (1999 to 2003) 

These responsibilities emphasized research design and implementation, capturing and analysis of data, and 
reporting of findings directed toward enhancing the mission of the company. These were achieved through 
the developing and communicating useful infonnation to clients, provision of direction and support to the 
research division, linkage of research and evaluation to communications, marketing, and organizational 
learning. 

Walter R. McDonald & Associates, Inc,, Centennial, CO 

Vice President for Research (2003 to 2007) 

As a participant with the management of the company the primary responsibilities were to oversee the 
research and evaluation functions and services of the organization. This position involved oversight for the 
company’s research projects and senior research staff, client relations, and participation in strategic 
planning and implementation. 


Child Protective Services Research Center, American Humane Association, Englewood, CO 
Vice President and Founding Director (2007 to 2012) 

Dr. Fluke rejoined American Humane as the center’s founder and director. Established with private 
funding, the purpose of the research center is to address long-standing issues related to the improvement 
of public child protective services. 

Children’s Innovation Institute, American Humane Association, Englewood, CO 

Vice President (2012) 
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In this capacity Dr. Fluke’s research work focused on fundamental services to improve the status of 
children and families including the formation of evidence-based policy and practices. 

GRANT AND CONTRACT SUPPORT: 

Current Grant and Contract Support 

• Pan Northern Data Project, Health & Social Services, Territorial Social Programs, Northwest 
Territories (2015-): The Public Health Agency of Canada (PHAC) Child Maltreatment section 
within the Centre for the Prevention of Chronic Diseases is responsible for addressing child 
maltreatment surveillance in Canada. PHAC is exploring using child welfare administrative data as an 
additional component to inform child welfare and family violence surveillance. The challenges of 
extracting data from a series of disparate systems has constrained PHAC and other entities from 
developing integrated regional and national administrative surveillance. Current circumstances 
provide a unique opportunity for collaboration in the development of systems and surveillance in a 
pro-active and planned manner. An inter-jurisdictional project has been developed to enable the 
Northern tier territories (Nunavut, North West Territories, and Yukon) to collectively assess data 
requirements and explore the development of a common pan-northern administrative dataset. Dr 
Fluke is project consultant. 

• Tennessee’s Department of Children’s Services (TNDCS), Title IVE Waiver Evaluation (July 2014- 
present) 

Chapin Hall at the University of Chicago, Vanderbilt University, and the Kempe Center are responsible 
for evaluating several initiatives for the TNDCS IVE waiver. These initiatives include the use and 
application of the Child and Adolescent Needs and Strengths (CANS) and CANS variants, advanced 
training for supervisors, and work to strengthen foster care families. Kempe staff are designing and 
evaluating child safety outcomes, as well as, surveying and analyzing workforce characteristics. Dr. Fluke 
is the Kempe project director. 

• Colorado Community Response (CCR) Evaluation, Office of Early Childhood, Colorado 
Department of Human Services (July 1, 2014- present) 

In 2013, the Offices of Early Childhood and Children, Youth, and Families received funding for the 
development of Community-based Child Abuse Prevention Services, for the Colorado Community 
Response (CCR) program. As part of a group of cornerstone prevention programs fonned or expanded 
under Governor Hickenlooper’s “Keeping Kids Safe and Families Healthy 2.0” initiative, families who 
have been referred to the child welfare system for alleged child maltreatment and whose referral was 
"screened out" and/or closed after initial assessment by local child protective services are eligible to 
voluntary enroll in the CCR program. In 2014 the evaluation was awarded to a consortium of universities 
headed by the School of Social Work at Colorado State University. The focus of the evaluation is to 
develop the ground work for evidence of the effectiveness of the program. The Kempe Center at the 
University of Colorado SOM has the lead for the CCR evaluation. Dr. Fluke is the Kempe Center PI under 
the CSU consortium PI 
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• Idaho’s Department of Health and Welfare (IDHW), Title IVE Waiver Evaluation (October 1, 2014- 
present) 

The Social Research Institute (SRI), College of Social Work, and the Department of Economics at the 
University of Utah, the Kempe Center for the Prevention and Treatment of Child Abuse and Neglect, 
Department of Pediatrics, University of Colorado School of Medicine, are partnering to conduct an 
evaluation of Utah’s Title IV-E waiver. The purpose of the six year evaluation of the waiver is to 
determine whether implementation the efficacy and effectiveness of Family Group Decision Making 
(FGDM) services minimize placement of children into DHW custody. Dr. Fluke is a member of the 
Kempe project team. 

• Utah’s Division of Child and Family Services (DCFS), Title IVE Waiver Evaluation (July 1, 2013- 
present) 

The Social Research Institute (SRI), College of Social Work, and the Department of Economics at the 
University of Utah, the Kempe Center for the Prevention and Treatment of Child Abuse and Neglect, 
Department of Pediatrics, University of Colorado School of Medicine, and the Fouis de la Part Florida 
Mental Health Institute (FMHI), University of South Florida are partnering to conduct an evaluation of 
Utah’s Title IV-E waiver. The purpose of the six year evaluation of the waiver is to determine whether 
implementation the efficacy and effectiveness of in-home services is improved, as well as to minimize 
placement of children into the custody of DCFS. Dr. Fluke is the Kempe Project Director. 

• Research Triangle Institute (RTI) Subcontractor (October 1, 2013- present) 

Under RTFs contract with the USHHS Assistant Secretary for Planning and Evaluation (ASPE) this 
projects calls for use of the National Child Abuse and Neglect Data System (NCANDS) and National 
Study of Child and Adolescent Wellbeing (NSCAW) II data to help determine how intervention type 
(Alternative vs. Investigative Response) selection is associated with safety-related outcomes for children. 
Dr. Fluke is the Kempe Project Director. 

• Chapin Hall, Foster Care Redesign for the Texas Department of Family and Protective Services 
(2010-present): The Texas Department of Family and Protective Services rolling out a modified financial 
system for reimbursing providers of out of home services in order to leverage funds to improve services 
aimed at creating settings that meet children’s needs and that shorten their time in care to planned exits 
without comprising safety or well being. Under sub-contract to Chapin Hall, Dr. Fluke is the Principal 
Investigator for American Humane Association. 

Selected Previously Funded Grant and Contract Support 

• U.S. Health and Human Services, Children’s Bureau, No Place Like Home: Family Group Decision 
Making for Children and Families Receiving In-Home Services (2011-2015): A federal grant awarded to 
the American Humane Association (AHA), Casey Family Programs (CFP) and three innovative child welfare 
agencies with mature family group decision making (FGDM) programs to test the effectiveness of FGDM in 
safely preventing children from entering or re-entering foster care when they are receiving in-home services. 
The project sites are Farimer County Department of Human Services (FDHS), Colorado; South Dakota 
Department of Social Services (SDDSS); and Texas Department of Family and Protective Services (TDFPS). 
All three sites will participate in a rigorous evaluation (experimental or quasi-experimental), longitudinal 
designs addressing FGDM process, outcomes and cost effectiveness. Dr. Fluke is the Principal Investigator. 
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• Texas Department of Family and Protective Services (TDFPS), Child Welfare Disparities Initiative 
Evaluation (2007-2008): This project entailed supporting TDFPS evaluate its initiative to identify sources 
of disparities, develop policy, program, and community responses to address disparities, and assess 
outcomes to detennine if outcomes in disparities were impacted. This contract was awarded to Walter R. 
McDonald and Associates, Inc. (WRMA) and entailed support for the analysis of TDFPS administrative 
data. The consultation also extended to implementing the evaluation utilizing the Decision Making 
Ecology framework as a basis for hypotheses. Dr. Fluke was the Principal Investigator for WRMA. 

• Center for Mental Health Services (CMHS), Substance Abuse and Mental Health Services 
Administration (SAMHSA), Comprehensive Community Mental Health Services for Children and 
Their Families Program, Phase V (2005 - 2007): This 5 year national evaluation involving 25 sites was 
intended to describe the children and families served, including behavioral, emotional, and functional 
characteristics; determine how children and their families change over time; document the services 
delivered and the effects and costs of these services; evaluate how systems of care develop over time; 
investigate the relationship between system change and child and family outcomes; document cultural and 
linguistic competence; examine the nature and effects of family-driven and youth-guided care; evaluate 
the effectiveness of factors influencing the implementation of systems of care and evidence-based 
practices within systems of care; develop evaluation capacity and quality monitoring in system of care 
communities; and identify factors influencing the sustainability of these programs. Dr. Fluke was the 
Principal Investigator for the project. 

• Texas Department of Protective and Regulatory Services, Texas Title IV-E Adoption Assessment 
Waiver Demonstration Project Cost-Effectiveness Analysis Project, (2001-2003): This project was to 
assess the cost-effectiveness of its title IV-E waiver on adoptions. The project linked with the overall 
evaluation of the project and utilized performance measures and cost data that were derived, as well as 
administrative data resources. Dr. Fluke was the Principal Investigator responsible for all aspects of the 
study and developed an administrative-level computer simulation model to support administrative 
assessment of program efficacy and in forecasting the impacts of changes in services due to the 
introduction of new policy. This was one aspect of an on-going multi-year research project of the 
Department, funded in part by a research grant from the HHS Children’s Bureau. 


SELECTED PUBLICATIONS: 

Fluke, J., Shustennan, G., Hollinshead, D., & Yuan, Y.T. (2008). Longitudinal analysis of repeated child 
abuse reporting and victimization: multistate analysis of associated factors. Child Maltreatment, 13 
(1), 76-88. 

Fluke, J. (2009). Allegory of the cave: On the theme of substantiation. Child Maltreatment, 14, 69-72. 

Fluke, J., Chabot, M., Fallon, B., MacLaurin, B., & Blackstock, C. (2010). Placement decisions and 
disparities among aboriginal groups: An application of the decision making ecology through multi¬ 
level analysis. Child Abuse & Neglect, 34 (1), 57-69. 

Detlaff, A., Rivaux, S., Baumann, D. J., Fluke, J.D., & Rycraft, J. R. (2011), Disentangling 

substantiation: the influence of race, income, and risk on the substantiation decision in child welfare. 
Children and Youth Services Review, 33 (9), 1630-1637. 
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Baumann, D.J., Dalgleish, L., Fluke, J., Kern, H. K. (2011). The Decision Making Ecology. American 
Humane Association: Denver. 

Gilbert, R., Fluke, J., O’Donnell, M., Gonzalez-Izquierdo, A., Brownell, M., Gulliver, P., Janson, S., & 
Sidebottom. (2012). Trends in child maltreatment in six developed countries. Lancet. 379(9817):758-72. 

Fluke, J., Merkel-Holguin, L. and Schene, P. (2013). Thinking differentially: a response to issues in 
differential response. Research on Social Work Practice, doi: 10.1177/1049731513481390. 

Merkel-Holguin, L., Hollinshead, D., Hahn, A., Casillas, K., and Fluke, J. (2014). The Influence of 

Differential Response and Other Factors on Parent Perceptions of Child Protection Involvement. Child 
Abuse and Neglect 39 18-31. doi:10.1016/i.chiabu.2014.11.008 

Graham, J.C., Baumann, D. J., Dettlaff, A.J., and Fluke, J.D. (2015). The Decision Making Ecology of 
Placing a Child into Foster Care: A Structural Equation Model, Child Abuse and Neglect. 49 12- 
23. DOI: 10.1016/j.chiabu.2015.02.020 

Detlaff, A.J., Graham, J.C., Holzman, J., Baumann, D. J., and Fluke, J.D. (2015). Factors that influence 
the removal decision in child protective services: development of an instrument to understand the 
decision-making process. Child Abuse and Neglect. 49 24-34. DOI: 10.1016/j.chiabu.2015.04.007. 

Williams, J., Merkel-Holguin, L., Allan, H., Maher, E., Fluke, J., and Hollinshead, D., (2015), Factors 
associated with staff perceptions of the effectiveness of family group conferences. Journal of the 
Society for Social Work and Research. 

Fallon, B., Chabot, M., Fluke, J., Blackstock, C., Sinha, V., Allan, K. and MacLaurin, B. (2015). 

Exploring alternate specifications to explain agency-level effects in placement decisions regarding 
aboriginal children: Further analysis of the Canadian Incidence Study of Reported Child Abuse 
and Neglect Part C. Child Abuse and Neglect. 49 97-106. DOI: 10.1016/j.chiabu.2015.04.012. 

Wittenstrom, K., Baumann, D. J., Fluke , J.D., Graham, J.C. (2015). The impact of drugs, infants, single 
mothers, and relatives on reunification: A Decision-Making Ecology approach. Child Abuse and 
Neglect. 49 86-96. DOI: 10.1016/j.chiabu.2015.06.010. 

Hollinshead, D., Kim, S., Fluke, J., and Merkel-Holguin, L. (2015). The Influence of Family, Agency, and 
Caseworker Dynamics on Caregivers’ Satisfaction with their Child Protective Services 
Intervention. Journal of Public Child Welfare. DOI: 10.1080/15548732.2015.1091762 

Lopez, M., Fluke, J., Benbenishty, R. and Knorth, E. J. (2015). Commentary - Decision making and 

judgments in child maltreatment prevention and response: An overview. Child Abuse and Neglect. 
49 1-11. DOI: 10.1016/j.chiabu.2015.08.013. 

Leeb, R. T. & Fluke, J.D. (2015). Commentary: Child maltreatment surveillance: enumeration, 

monitoring, evaluation, and insight. Health Promotion and Chronic Disease Prevention in Canada: 
Research, Policy and Practice. 35(8/9). 


6 











Joyce James Consulting 

Equal Treatment Does Not Lead To Equity 


Curriculum Vitae 

Joyce M. James, LMSW-AP 
2401 Silent Brook Trail 
Round Rock, TX 78665 
jjameslmsw@aol.com 
512-244-1913 


Professional Experience 

2013 -Present Racial Equity Consultant, Joyce James Consulting (JJC). Principal owner and 
senior racial equity consultant. Provide executive-level consultation, technical 
assistance, coaching, leadership development, facilitate focus groups, community 
engagement strategies, data analysis, strategic planning, and training to key 
leaders, staff, and community stakeholders from a cross section of governmental 
and non-governmental entities and organizations with a specific focus on equity 
in outcomes for all populations. Nationally recognized as a leading expert on 
issues impacting the most vulnerable populations, Ms. James professional career 
has spanned more than 35 years. 

2014- Present Consultant, Fort Worth Independent School District, Provide racial equity 
training, data analysis, strategic planning, facilitate focus groups, leadership 
development and technical assistance to District’s Leadership and specific 
campuses to reduce racial inequities in student outcomes. 

2014-2015 Consultant Child Advocates of Indianapolis Indiana, Provide racial equity 

training and technical assistance to staff and volunteers and work in partnership 
with faith communities and Child Advocates to implement JJC Ministry of Caring 
Model for recruitment of African American CASA Volunteers to reduce 
disproportionality and improve outcomes for all children in care of the Courts in 
Marion County. 

2015 Consultant Allegheny County Department of Children and Family Services, 

Provide technical assistance and work in partnership with Allegheny DCS 
Leadership to identify racial inequities, engage and orient internal and external 
stakeholders in understanding the underlying causes of disproportionality and 
disparities. Conduct focus groups to gather constituent input. Review policy and 
practice models that may contribute to disproportionality and disparities. Assist in 
the development and implementation of strategies for reducing and ultimately 
eliminating inequities. 
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2014-2015 Consultant, New York City Department of Children and Family Services, Provide 
racial equity and leadership development training. 

2014 Consultant, Austin Community College, Provide technical assistance, leadership 

development, focus groups to assess the internal culture and increase knowledge 
of leadership to examine internal and systemic factors that contributed to racial 
inequities for Males of Color. 

2014 Consultant, College of the Mainland, Provide technical assistance, leadership 

development, focus groups to assess the internal culture and increase knowledge 
of leadership to examine internal and systemic factors that contributed to racial 
inequities for Males of Color. 

2010-2013 Associate Deputy Executive Commissioner, Texas Health and Human Services 
Commission Center for Elimination of Disproportionality and Disparities and 
State Office of Minority Health. The Center was created out of recognition Ms. 
James’ strong and effective leadership in the area of systems reform in the Texas 
Child Protective Services System, and a desire to expand the highly effective 
Texas Model for Addressing Disproportionality, created under her leadership, to 
all systems that operate under the umbrella of the Texas Health & Human 
Services Commission. In 2011, the Center was established in statute by Senate 
Bill 501 of the 82nd Legislature authorizing the Center as the State Office of 
Minority Health and charging it with addressing health disparities in all Health 
and Human Services agencies. SB 501 also established an Interagency Council on 
addressing Disproportionality with Ms. James as Chair. The role of the Council 
was to examine and make recommendations to address disproportionality in child 
welfare, juvenile justice, education, and mental health, and to submit a report with 
its findings and recommendations to the Texas Legislature. 

2013 Consultant Texas A&M University Office of Diversity, Provide cross systems 

racial equity workshop to increase awareness and understanding of institutional 
and structural racism in outcomes for poor communities and communities of 
color. 

2011 Consultant, State of Connecticut, Department of Children and Families. 

Consultation and advice on identifying and addressing disproportionality through 
a data-driven, community engagement approach. Services perfonned included 
technical assistance and guidance on tying cultural competency training to 
strategic goals to eliminate disproportionality 

2011 Consultant, State of Washington, Child Protective Services. Consultation and 

advice on identifying and addressing disproportionality through a data-driven, 
community engagement approach. Services performed included small- and large- 
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group training of a broad cross section of employees and community 
representatives on reducing disproportionality and disparities. 

2010 Consultant, State of Ohio, Child Protective Services. Consultation and advice on 

identifying and addressing disproportionality through a data-driven, community 
engagement approach. Services perfonned included a broad range of technical 
assistance and consultative services aimed at supporting the organizational and 
strategic goals. 

2009 - 2010 Deputy Commissioner, Texas Department of Family and Protective Services 
(DFPS). Provide leadership for 11,000 DFPS employees in the day-to-day 
operations of the agency’s Child and Adult Protective Services Programs and the 
Residential and Child Care Licensing Programs. Oversee and manage the Center 
for Learning and Organizational Effectiveness, Center for Policy& Research, and 
Center for External Affairs and Communication. 

2004 - 2009 Assistant Commissioner, Texas Department of Family and Protective Services, 
Child Protective Services. Provide leadership and direction for the State 
administered child welfare system’s 254 counties and approximately 9,000 staff. 
Led and oversaw the implementation of Senate Bill (S.B.) 6 and S.B. 758 
resulting in Texas being the first State to enact legislation requiring CPS to 
address Disproportionality and the allocation of unprecedented resources for 
implementation of new child welfare practice models and an additional 3,500 
staff. Spearheaded numerous nationally known research projects involving Risk 
Assessment, Family Group Decision Making, Kinship Care, Decision-Making 
and Disproportionality. Provided the leadership and overall vision that resulted in 
significant gains in overall improvements for all populations served including a 
reduction in disproportionality and disparities for African American and Native 
American children, youth, and families. 

2009 -2010 Consultant, State of Colorado, Disparities Resource Center. 

Consultation and technical assistance on identifying and addressing 
disproportionality through leadership development and a data-driven, community 
engagement approach. 

2009 Consultant, State of New York, Child Protective Services, Albany, NY. 

Consultation and technical assistance on identifying and addressing 
disproportionality through a data-driven, community engagement approach. 

2008 Consultant, State of Kentucky, Cabinet for Health and Family Services. 

Consultation technical assistance on identifying and addressing disproportionality 

Honors and Awards 
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2011 First recipient of the annual Joyce James Courageous Leadership 

Award, in recognition of exceptional vision, courageous leadership, and 
unwavering commitment toward eliminating disproportionality and disparities; 
Texas Statewide Disproportionality Task Force Committee Chairs 

2011 Texas President’s Award, for boldly accepting responsibility to address vast 

health inequities endured by minorities throughout the state of Texas; National 
Association for the Advancement of Colored People (NAACP) 

2010 Leading the Movement in Texas to Eliminate Disproportionality and Disparities; 

Casey Family Programs Texas Strategic Consulting 

2010 Moore-Bowman Award, for outstanding contributions to family wellbeing in 

Texas; Texas Council on Family Relations 

2009 Betsey R. Rosenbaum Award for Excellence in Public Child Welfare 

Administration; National Association of Public Child Welfare Administrators 
(NAPCWA) 

2009 Charles Edwards Award, recognizing outstanding leadership, guidance, and 

counsel; Association of Administrators of the Interstate Compact on the 
Placement of Children 

2007 Recognizing Outstanding Leadership and Service in the prevention of child abuse 

and neglect; US Department of Health and Human Services, Administration on 
Children and Families 

2007 Commissioner’s Award Recipient, approved by the Governor of Texas; presented 

by the Administration for Children and Families 

2006 Valeria Bullard Black Administrator of the Year; Black Administrators in Child 

Welfare Annual Conference 

2006 Director’s Award; Family Preservation Institute of New Mexico State University 

2005 YWCA Woman of the Year Award for Government Service/ Public Policy; 

Young Women’s Christian Association 

2002 Woman of Vision, Reflection and Action Award, National Association of Negro 

Business and Professional Women’s Club 
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1998 

Service 

2015 

2015 

2014 

2014 

2011 -2013 

2011 -2012 

2011 -2013 

2011 -2013 
2011 -2013 
2009-2010 

2009-2010 

2009-2010 

2008-2012 

2008-2010 

2007-2013 


Outstanding Women in Texas Government; Texas Department of 
Protective & Regulatory Services, awarded by George Bush, Governor of Texas 


National Council on Accreditation Public Agency Committee 

Texas Court Appointed Special Advocates, Board Member 

National Black Administrators in Child Welfare, Board Member 

Partners in Austin Transforming Health, Steering Committee 

Interagency Council on Addressing Disproportionality and Disparities; 

Co-Chair established by Senate Bill 501, 82nd Texas Legislature, Regular Session 

Juvenile Justice Advisory Committee on Breaking Schools’ Rules, appointed by 
Supreme Court Chief Justice Wallace Jefferson 

Transfonning Texas: Healthy People in Healthy Communities, Transfonning 
Texas Leadership Team 

Judicial Advisory Committee on addressing Disproportionality in the Court; 

Texas Healthy Babies Expert panel 

Council on Children and Families Co-Chair; established by Senate Bill 1646, 81st 
Texas Legislature, Regular Session, 2009 

Task Force on Strengthening Nonprofit Capacity, Chair, established by House 
Bill 492, 81st Texas Fegislature, Regular Session, 2009 

Task Force for Children with Special Needs, established by Senate Bill 1824, 81st 
Texas Legislature, Regular Session, 2009 

Mountain and Plains Child Welfare Implementation Center Board of Directors 

National Association of Public Child Welfare Administrators Executive 
Committee 

Supreme Court of Texas Judicial Commission on Children, Youth, and Families; 
appointed by Supreme Court Chief Justice Wallace Jefferson 
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2006 - 2009 University of Kentucky, Quality Improvement Center on the Privatization of 
Child Welfare Board of Directors 

2005 - 2009 Children’s Advocacy Centers of Texas Board of Directors 
1996 - 2013 Texas Center for the Judiciary Children’s Justice Act Task Force 

Publications/ Books 

James, Joyce, et al. “Addressing Disproportionality and Disparities in Health and human 
Services Systems, Columbia Press 2014 

James, Joyce, et al. "Disentangling Substantiation: The Influence of Race, Income, and Risk on 
the Substantiation Decision in Child Welfare." Children and Youth Services Review (2011). 
Print. 

James, Joyce, et al. "Evidence-Based Practice in Family Group Decision-Making for Anglo, 
African American and Hispanic Families." Children And Youth Services Review 31 .Integrating 
Cultural Competency and Evidence-based Practices in Child Welfare (2009): 1187-1191. Print. 

James, Joyce, et al. "The Intersection Of Race, Poverty And Risk: Understanding The Decision 
To Provide Services To Clients And To Remove Children." Child Welfare 87.2 (2009): 151- 
168. Print 

James, Joyce, et al” Addressing disproportionality through Undoing Racism, leadership 
development, and community engagement Child Welfare 87: 279-2966 

Education: 

BA Sociology, Lamar University, Beaumont, TX 1979 

Certifications: 

Licensed Master Social Worker- Advanced Practitioner 
Effective Black Parenting Instructor 


2401 Silent Brook Trail • Round Rock, Texas 78665 • 409-553-0314, 512-244-1913 • joyce james consulting.com • 

jjameslmsw@aol.com 


VITA 


Donald J. Baumann, Jr. 

ADDRESS: Work: Saint Edwards University 

3001 South Congress 
Austin Texas 78704-6489 
512-448-8648 
do n al db(<7;stcd wards. edu 



EDUCATION 

1981 Ph.D., Social Psychology, Arizona State University 

(emphasis on Applied Social/Community Psychology) 

1978 M.A., Social Psychology, Arizona State University 

1975 B.A., Magna Cum Laude, Arizona State University 

PROFESSIONAL EXPERIENCE 

Present: Associate, James Joyce Consulting, LCC 

2003-Present: Instructor, Saint Edwards University. 

2010-Present; Consultant on various projects pertaining to disproportionality 

2012 - 2013: Senior Research Fellow, American Humane Association 

1997 - 2010: Head of Program Evaluation and Research, Child Protective Services, Texas 
Department of Family and Protective Services. 

Research & Evaluation Activities: Principal Investigator on a number of grants funded through 
the Child Abuse and Treatment Act, Casey Family Programs and The American Humane 
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Association. The general purposes of the grants were program evaluation, decision-making and 
disproportionality. For example, one grant was used to implement and evaluate a state law 
requiring the agency to screen more serious cases into a child abuse investigation track and less 
serous cases into an assessment track. A second developed an actuarial model that allowed the 
agency to build into the Child Protective Services computerized data system that "alerted" 
caseworkers to the potential in a case for a future child fatality. Another large project involved 
understanding the disproportionate number of African Americans investigated and in foster care. 
Analyses involved accessing large-scale data sets (millions of records) from state systems and 
constructing secondary data sets from these data and interview and survey data accessed 
elsewhere. In a typical year, three to four large scale research and evaluation projects were 
funded by grants and ten moderate size projects funded through the department were conducted. 
Analyses were conducted using SAS, SPSS and other, packages. The typical types of analyses 
included descriptive work, multiple regression, logistic regression, neural network, structural 
modeling and multi-level modeling. The projects also required managing tight project timelines. 
Publications, reports and conference presentations resulted. In a typical year, three to four large 
scale research and evaluation projects and ten moderate size projects were conducted. The 
overarching purposes of the projects were to enable management to make programmatic 
decisions and to understand and aid caseworker decision-making. 

Supervision and Teaching: Recruited, trained, and supervised four Ph.D. Research Associates, 
five Program Specialists, one Masters level Research Assistant, two graduate students, six 
Research Assistants, and two undergraduate Independent Study students. The section typically is 
comprised of a team of seven to ten individuals at any one time as they work on a variety of 
projects. These include three major multi-year research projects, three single year research 
projects, and numerous ad hoc short-term projects. Additional supervision duties include 
oversight on contracts awarded for research projects conducted by independent organizations 
(typically a university). In addition to working closely with management and staff, supervision 
duties include quality assurance, research design, measurement development, data collection and 
analysis, publications, report writing and presentation. Taught at least one course per semester 
from 2000 to the present at St. Edward University. Classes include Research Methods, 
Abnormal Psychology, Personality Theory, Social Psychology, Developmental Psychology, 
Program Evaluation, and Tests and Measurement. 

Consultant on eight federally funded multi-year research grants. 

1997 - 1991: Lead Researcher, Program Evaluation & Research (formerly Program 
Assessment Section), Child Protective Services, Texas Department of Protective and 
Regulatory Services (formerly DHS). 

Research and Evaluation Activities: Co-principal Investigator on a three-year research grant 
funded by the Child Welfare Division of the Department of Health and Human Services. The 
purpose of the grant was to study caseworker decision-making and risk assessment. Structural 
and other models that used environmental, organizational, individual, and case factors were used 
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to predict shifts in judgment thresholds and decision consequences (e.g., child safety, burnout 
and turnover) were developed. Numerous data sets were developed. They required acquiring 
data from state systems using SQL and other programs, constructing data sets from these data 
and data from other sources (e.g., public, caseworkers, etc.), and conducting diagnostics and 
analyses on them through the use of SAS, SPSS and other packages (in particular survival, time 
series and other forecasting procedures were often used. The resulting report covered twelve 
chapters and a book on the topic is being negotiated. Many conference presentations resulted, 
one of which was a keynote address. 

Tested models of organizational perfonnance using techniques from operations research (e.g., 
linear programming/optimization etc.). This involved developing and simultaneously testing sets 
of system perfonnance equations. Worked on a number of smaller research studies that included 
workload management, worker perfonnance, and outcome measurement. Outcome measurement 
included being familiar with education-related issues. 

Began directing a three year Federally funded research grant funded through the Child Abuse and 
Treatment Act to develop empirical models that screened cases in and out of the system of care at 
various decision points (e.g., risk and safety instruments). 

Worked with numerous agencies and groups (i.e., shelter providers, foster parents, residential 
treatment facilities, etc. to help develop outcome measures that were used to begin building a 
quality assurance system that would allow performance feedback to the agencies and the 
department. The development of stakeholder satisfaction surveys was also part of this three-year 
process. This involved knowledge of the Texas education system and a familiarity with large- 
scale education data sets. 

Consultant to several states and agencies on decision-making and risk assessment. 

Supervision and Teaching: Recruited, trained, and supervised two Ph.D. Research Associates, 
one Masters level Research Assistant, three graduate students, four Research Assistants, and 10 
to 15 undergraduate Independent Study students. The section typically had a team of ten of these 
individuals at any one time as they worked on two major multi-year, and numerous single study, 
research projects. Individuals participated at every level of the projects from design (qualitative 
and quantitative) to data collection and analysis (e.g., descriptive, predictive, structural modeling, 
etc.), through report writing and verbal presentations to agency staff and management. 
Additionally, helped develop one of the early quality assurance systems in the division. Taught 
at least one psychology course per semester from 1993 to 1997 at Austin Community College. 


1989-1991 : Visiting Professor, Department of Psychology, Trinity University, San Antonio, 
Texas. 

Research and Evaluation Activities: Principle Investigator on a grant from the City of Houston, 
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Harris County, the United Way of the Texas Gulf Coast, and McKinsey and Company. The 
purpose of the grant was to conduct research and program planning on homelessness, in 
particular to produce an empirically based plan for coordinated services among a diverse service 
delivery system. In order to collect data, a methodology to obtain a representative sample of 
homeless individual on the street, in shelters, in abandoned buildings and temporarily housed 
with others had to be devised. This resulted in an estimation procedure presented to the U.S. 
Census Bureau. The research involved coordinating individuals collecting data from the 
homeless throughout the city of Houston and outlying areas as well as managing challenging 
timelines. A large-scale report and the distribution of seven million dollars to various agencies 
resulted. 

Research Director on a program evaluation of a three-year team case management demonstration 
project from the U.S. Department of Health and Human Services involving mental health and 
employment among homeless adolescents, women and adult men. Data collection involved 
acquiring data from state systems, constructing data sets from these data and data from other 
sources (e.g.,the homeless, providers, etc.), and conducting diagnostics and analyses on them 
through the use of SAS, SPSS and other packages. Numerous reports resulted. 

Supervision and Teaching: Recruited, placed with the City of Houston, trained and supervised 
ten graduate students from Texas Southern University, one from Saint Thomas University, and 
two from the University of Houston. The research/planning team worked with McKinsey and 
Company and other community leaders (e.g., the City Council, County Commissioners, board 
members, etc.) to produce a plan for the homeless. 

Recruited, trained, and supervised ten undergraduate students from the University of Texas who 
were part of a research team to study case managed job training for the homeless. Students 
worked with the Austin Women's Center, the Salvation Army, Middle Earth, the Private Industry 
Council, and the Texas Employment Commission as part of the Project. 

Taught courses in Introductory Psychology (four semesters), Social Psychology (two semesters), 
Developmental Psychology (two semesters), Motivation and Emotion (one semester), Personal 
Relationships (one semester), Social Problems (one semester), and Independent 
Readings/Research (two semesters). Directed and advised undergraduate students, participated 
with the faculty in curriculum development, and supervised numerous research projects on 
campus. 


1982-1989: Assistant Professor, Department of Psychology, University of Texas at Austin. 

Research and Evaluation Activities: Principle Investigator, Austin Police Department. Quasi- 
experimental investigations of time spent by police officers on chronic nuisance calls. In order to 
accomplish this, SAS was purchased by the police department, installed on the main-frame and a 
programmer who understood the data base was assigned to work with the Principle Investigator. 
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A process for reducing chronic nuisance calls resulted. 

Supervision of a Doctoral Dissertation of police decision making as a function of prior 
dispatched expectancies. Studies involved the retrieval of archival data from the Police 
Department's main-frame computer using SAS and the use of the police training academy. 

Principle Investigator, University of Texas. Theory and research involving rape cases based on a 
consent defense. Studies involving actual data from the Austin District Attorney's Office and 
simulated juror studies were conducted. Publications resulted. Analyses involved the use of 
SPSS. 

Principle Investigator, the Hogg Foundation and the City of Austin. The purpose of the two 
years of funding was to develop theory, conduct research, and provide program planning, and 
program evaluation on the homeless. A methodology was developed to collect a representative 
sample of street homeless, data were collected and both a model of homelessness and of service 
provision was provided. The work continued beyond the project for six years. Analyses 
involved the use of SPSS. Publications, reports, and conference presentations resulted. 

Principle Investigator, Policy Research Institute, University of Texas. The purpose of the grant 
was to investigate arrest rates among the mentally ill. This project involved demonstrating that 
visibility rather than criminalization caused heightened arrest rates among the homeless mentally 
ill. A report resulted. 

Principal Investigator, University of Texas. Programmatic research involving the issue of 
altruism vs. egoism. This laboratory research program extended from graduate school spanning 
eight years and resulted in a number of publications. 

Supervision and Teaching: Developed relationships with the Austin Police Department and the 
Travis County Prosecutor's office. Supervised graduate and undergraduate students working with 
these agencies. Students wrote theses and articles. 

Recruited, trained and supervised ten graduate students from Psychology, Sociology, and Social 
Work to conduct two years worth of research and planning on the homeless. Supervised and 
trained at least 5 graduates and ten undergraduate students from the Department of Psychology 
each semester. Worked with the Austin City Council, the Downtown Partners, the Salvation 
Anny, MHMR, the Health Department, the Battered Women's Center, local churches and private 
citizens in the process. Students wrote reports and articles. 

Taught undergraduate courses in Introductory Psychology (14 semesters), Social Psychology (six 
semesters), and Independent Study (14 semesters). Taught graduate courses in Applied Social 
Psychology (six semesters) where a large share of the course was devoted to program evaluation, 
quasi-experimental methods, and qualitative methods. Taught Special Topics at a graduate level 
on Altruism and Homelessness (one semester each). Recruited and advised undergraduate and 
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graduate students, wrote and secured grants, supervised internships, conducted and directed 
research, served on the Faculty Senate, the Department Budget Council, and departmental 
committees. 


1981 - 1982: Faculty Associate, Department of Education, Arizona State University. 

Research and Evaluation Activities: Project Director, qualitative and quantitative program 
evaluation of an academic program designed to provide tutoring and advisement to student- 
athletes at Arizona State University. Required knowledge of education-related issues and of 
large-scale educational data sets. 

Principle Investigator, City of Phoenix Police Department/Phoenix South Community Mental 
Health Center. Quasi-Experimental and qualitative program evaluation of crisis intervention 
services provided to individuals on the part of a police-citizen crisis intervention program. The 
principle dependent measures were number of police calls and time spent by police officers per 
call. Database consisted of over 10,000 police calls from a two year period and six months of 
observations of police and crisis intervention workers. 

Consultant on grant from the National Institute of Drug Abuse. The purpose of the grant was to 
investigate the effects of treatment upon heroin abusing males and females. 

Consultant on a contract with the State of Arizona. The purpose of this project was to develop 
and implement an outcome measurement instrument to be used by providers of various programs 
throughout the state of Arizona. 

Research Supervision and Teaching: Recruited, trained and supervised five graduate students 
from the Department of Education and two undergraduates from the Department of Psychology 
to study student athletes. Students worked with the University administration, Faculty Senate, 
and the Athletic Program. Students participated in qualitative and qualitative data collection and 
analysis and report writing. 

Recruited, trained and supervised ten undergraduate and one graduate student who participated in 
observations of police officers and crisis intervention workers on the job. Qualitative and 
quantitative data were collected, and analyzed. Students also wrote reports and articles. 

Taught courses in Social Psychology (two semesters) and Independent Study (two semesters). 
Advised undergraduate and graduate students. Wrote and secured grants, supervised internships, 
conducted and directed research. 


1976-1981: Graduate Student, Arizona State University. 
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Research and Evaluation Activities: Research Assistant, Phoenix South Community Mental 
Health Center. Directed program evaluation and development, provided methodological and 
statistical consultation, and directed the development of computerized research data base 
systems. Additional duties included development and supervision of theoretical as well as 
practice models for rates under treatment studies, fee collections studies, etc. 

Research Assistant with a multidisciplinary team that sought to better understand literacy 
development in the Community College. Research involved investigating the relationship 
between stress and the use of social support systems by minorities in the Community College. 
Duties included instrument development and implementation, staff training and management and 
statistical analysis of data. 

Planner, Migrant Opportunity Program. Phoenix, Arizona. Duties included, grant writing, 
program planning and evaluation, and staff training and development. 

Research Analyst, state-funded career education survey, Career Education Evaluation Project, 
Arizona State University. Duties included conceptual and statistical consultation. 

Research Analyst, Alcohol Treatment Programs, Veterans Administration Hospital, Phoenix, 
Arizona. Developed and experimentally tested an attribution model of therapeutic 
intervention. 

Project Director, State-funded needs assessment survey, Community Behavioral Health Services, 
Phoenix West Central Catchment area. Planned, developed and tested a needs assessment model 
based on primary prevention. 

Supervision and Teaching: Taught courses in Social Psychology (four semesters), Introductory 
Psychology (one semester), Developmental Psychology (one semester), and Personality Theory 
(one semester). The latter three courses were taught at a community college. Supervised and 
trained undergraduate students and conducted research. Provided field placements for students in 
a mental health center and a farmworker program. 


PUBLICATIONS: 

Kenrick, D., Baumann, D., & Cialdini, R. (1979). A step in the socialization of altruism as 
hedonism: Effect of negative mood on children's generosity under public and private conditions. 
Journal of Personality and Social Psychology, 37, (5), 747-755. 

Christiansen, J.A., Reich, J.W., Obitz, F.W., & Baumann, D. (1980).Effect of infonnation about 
role and outcome on alcoholic’s attributional processes. Journal of Studies on Alcoholism, 
11,922-929. 
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Baumann, D., Cialdini, R., & Kenrick, D. (1981). Altruism as hedonism: Helping and self¬ 
gratification as equivalent responses. Journal of Personality and Social Psychology, 40, 1039- 
1046. 


Cialdini, R., Baumann, D., & Kenrick D. (1981). A new unobtrusive measure of attitude. Social 
Psychology Quarterly, 44, 254-259. 

Cialdini, R., Baumann, D., & Kenrick, D. (1981). Insights from sadness: three-step model of the 
development of altruism as hedonism. Developmental Review, 1, 207-223. 

Cialdini, R., Kenrick, D., & Baumann, D. (1982). Effects of mood on prosocial behavior in 
children and adults. In N. Eisenberg (Ed.), Development of Prosocial Behavior. New York: 
Academic press. 

Baumann, D., Obitz, F., & Reich, J. (1983). Attribution theory: A fit with problems of 
substance abuse. International Journal of the Addictions . 

Baumann, D., Cialdini, R. & Kenrick, D. (1983). Mood and sex differences in the development 
of altruism as hedonism. Academic Psychology Bulletin, 5, 2, 299-307. 

Baumann, D.J. (1984). Decision with consequences (Review of The Badge and the Bullet: 
Police Use of Deadly Force). Contemporary Psychology, 29, 557-558. 

Manucia, G., Baumann, D. & Cialdini, R. (1984). Mood influences on helping: Direct effects or 
side effects? Journal of Personality and Social Psychology, 46, 2, 357-364. 

Okun, M.A., Kardosh, C.A., Stock, W.A., Sandler, I.N., & Baumann, D. (1986). Measuring 
perceptions of the quality of academic life among college students. Journal of College Student 
Personnel, 2J_, 5, 447.451. 

Baumann, D., Schultz, D.F., Brown, C., Paredes, R., & Hepworth, J. (1987). Citizen 
participation in police crisis intervention activities. American Journal of Community 
Psychology. 15, 4, 459-471. 

Baumann, D., & Grigsby, C.G. (1988). Understanding the homeless: From research to action. 
Hogg Foundation for Mental Health, University of Texas. 

Okun, M.A., Baumann, D., & Sandler, I.N. (1989). Ethnicity, source of informal support, and 
buffer and booster effects for school events. American Journal of Community Psychology. 

Grigsby, C.G., Baumann, D., Gregorich, S. & Roberts-Gray, C. (1990). Disaffiliation to 
entrenchment: A model for understanding homelessness. In B. Shinn (Ed). Journal of Social 
Issues: Special Edition on Homelessness, 46, 4, 1414-156. 
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Baumann, D., Roberts-Gray. C., Grigsby, C. (1991). Estimates of the homeless in Houston, 
Texas. In M. Taeuber (Ed), Conference proceedings for enumerating homeless persons: 
Methods and data needs. U.S. Bureau of the Census, Washington, D.C., pp. 177-178. 

Baumann, D., & Holahan, C.J. (1992). The community psychologist. In Gifford, R. (Ed). 
Psychology in Everyday Life: An Introduction to the Practices of Psychology. Allyn & Bacon: 
Boston. 

Gains, S.O., Roberts, D., & Baumann, D. (1993a). When all the World's a Stage: The impact of 
events on news coverage of South Africa. Journal of South African Studies. 

Osborne, R., E., Karlin, J. E., Baumann, D., Osborne, M., and Nelms, D. (1993b). Perceptions of 
health on the part of the homeless. Journal of Social Distress and the Homeless, Vol 2, No 2, . 

Kem, H., Baumann, D., & Sheets, D. (1994). Texas WISDOM: A three-year research project to 
build effective CPS decision making instruments. In Tatara, T. (Ed). Seventh National 
Roundtable on Risk Assessment: Summary of Highlights. American Public Welfare Association, 
Washington, D.C. 

Fluke, J., Baumann, D., England, P., Kern, H., McFadden, T., and Schultz, D.F. (1995). 
Emerging critical conceptual issues in risk assessment research and practice. In Tatara, T. (Ed). 
Eighth National Roundtable on Risk Assessment: Summary of Highlights. American Public 
Welfare Association, Washington, D.C. 

Baumann, D. & Sheets, J. (1997). Foundations of the decision making ecology and patterns of 
decision making. In Tatara, T. (Ed). Eight National Roundtable on Risk Assessment: Summary 
of Highlights. American Public Welfare Association, Washington, D.C. 

Baumann, D. (1997). Decision theory and CPS decision making. In Morton, T.D. and Holder, 

W. (Eds). Decision Making in Children’s Protective Services: Advancing the State of the Art. 
National resource Center on Child Maltreatment, Atlanta, pp 17-37. 

Baumann, D., Sheets, J., Graham, J.C., Gober, K., and Reid, G. (1999). Concept guided risk 
assessment. In Edwards, M. (Ed). Tenth National Roundtable on Risk Assessment: Summary of 
Highlights. American Humane Association, Denver, CO 

Caouette, K. & Baumann, D. (2000). Texas 2000; The evolution of risk assessment in Texas. In 
Edwards, M. (Ed). Twelfth National Roundtable on Risk Assessment: Summary o f Highlights . 
American Humane Association, Denver, CO 

Fluke, J., Graham, J.C., Baumann, D. & Esterline, J. (2001). System dynamics simulation of 
child welfare permanency outcomes: An application of the Children's Sendees Simulation 
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Model. In Hernandez, M. & Hodges, S. (Ed). Brooks, Baltimore 

Baumann, D., Graham, J.C., Beauvais, C., and Bradstreet, R. (accepted with revisions). Moral 
evaluations and the law: A prototypical case. Journal of Applied Social Psychology. 

Baumann, D. J., (1997). Decision theory and CPS decision making. In Morton, T.D. and Holder, 
W. (Eds). Decision Making in Children’s Protective Services: Advancing the State of the Art L 
National resource Center on Child Maltreatment, Atlanta, pp 17-37. 

Baumann, D., Kern, H.. & Fluke, J. (1997). Worker Improvements to the Decision and Outcome 
Model (WISDOM): The Child Welfare Decision Enhancement Project. Prepared for the 
Department of Health and Human Services, Child Welfare Division, Washington, D.C. 

Fluke, J., Graham, J.C., Baumann, D. J., & Esterline, J. (2001). System dynamics simulation of 
child welfare permanency outcomes: An application of the Children's Sendees Simulation 
Model. In Hernandez, M. & Hodges, S. (Ed). Brooks, Baltimore 

Baumann, D.J., Law, J.R., Sheets, J., Reid, G., & Graham, J.C (2005). Evaluating the 
effectiveness of actuarial risk assessment models. Children and Youth Services Review f21, 465- 
490. 

Baumann, D.J., Law, J.R., Sheets, J., Reid, G., & Graham, J.C (2006). Remarks concerning the 
importance of evaluating actuarial risk assessment models: A rejoinder to Will Johnson. 
Children and Youth Sendees Review ,_28, 715-725. 

Rivaux, S. L., James, J., Wittenstrom, K., Baumann, D. J., Sheets, J., Henry, J., and Jeffries, V. 
(2008). The intersection of race, poverty and risk: Understanding the decision to provide 
services to clients and to remove children. Child Welfare, Special Issue on Disproportionality. 

Sheets, J., Wittenstrom, K., Fong, R., James, J., Tecci, M., Baumann, D. J., and Rodriquesz, C. 
(2009). Evidence based practice in family group decision-making for African American and 
Hispanic Families. Children and Youth Services Review, 31, 1187-1191. 

Graham, J. C., Stepura, K., Baumann, D. J., and Kern, H., (2010). Predicting fatalities among 
less severe investigations. Children and Youth Sendees Review, 32, 274-280. 

Baumann, D. J., Fluke, J. D., and Dalgleish, L., (2011a). The Decision Making Ecology. Special 
Monograph: The American Humane Association. 

Dettlaff, A. J., Rivaux, S. R., Baumann, D. J., Fluke, J. D., Rycraft, J. R., & James, J. (2011). 
Disentangling substantiation: The influence of race, income, & risk on the substantiation decision 
in child welfare. Children and Youth Services Review, 33(9), 1630-1637. 
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Baumann, D.J., Grigsby, C., Sheets, J., Reid, G., Graham, J.C., Robinson, D., Holoubek, J., 
Farris, J., & Jeffries, V.. (2011c). Concept guided risk assessment: Promoting prediction and 
understanding. Children and Youth Services Review, 33, 1648-1657. 

Wang, E. W’, Lambert, M. C., Johnson, L. E., Boudreau, B., Breidenbach, R., and Baumann, D. 
J. (2012). Expediting permanent placement from foster care settings: The role of family group 
decision-making. Children and Youth Services Review, 34, 845-850. 

Baumann, D. J., Fluke, J. D., Dalgleish, L., & Kern, H. (2014). The Decision Making Ecology. 

In A. Shlonsky & R Benbenishty (Eds.). From Evidence to Outcomes in Child Welfare: An 
International Reader, (pp 24-40). New York, NY: Oxford University Press. 

Fluke, J. D., Baumann, D. J., Dalgleish, L., & Kern, H. (2014). Decisions to Protect Children: A 
Decision Making Ecology. In Korbin, J. E., and Krugman, R. D. (Eds.). Handbook of Child 
Maltreatment, (pp.464-476). New York, NY. Springer Publishing. 

Dettlaff, A.J., Graham, J.C., Holzman, J., Baumann, D.J., & Fluke, J.D. (2015). The 
development of an instrument to understand child protective services decision-making, with a 
focus on placement decisions. Child Abuse and Neglect. 
http://dx.doi.Org/10.1016/i.chiabu.2015.04.007 

Graham, J. C., Dettlaff, A., Baumann, D. J., & Fluke, F. D., (2015). The decision-making 
ecology of placing children in foster care: A structural equation model. Child Abuse and Neglect. 
http://dx.doi.Org/10.1016/i.chiabu.2015.02.020 

James, J.., Rodriguez, C., Craig, S., & Baumann, D.J., (accepted). Reform Efforts in Texas: The 
Community Engagement Model and Preliminary Evaluation Findings. Protecting Children, The 
American Flumane Association. 

Wittenstrom, K., Baumann, D.J., Fluke, J. D., Graham, J.C., & James, J. (2015). The impact of 
drugs, infants, single mothers and relatives on reunification: A Decision-Making Ecology 
approach. Child Abuse and Neglect. http://dx.doi.Org/10.1016/i.chiabu.2015.06.010 


TECHNICAL REPORTS (Selected reports after 2002 shown. Others available on 
request) 

Kenrick, D.T., Braver, S.L., Baumann, D., & Paredes, R. (1981). Form for assessment of Client 
Treatment Services . Prepared for State of Arizona, State Director of Behavioral Health Services. 

Baumann, D., Brown, C.B., & Schultz, D.F. (1982). Phoenix South Community Mental Health 
Center Family Stress Program: Final Evaluation Report . Prepared for the Phoenix Police 
Department. December. 
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Baumann, D., & Gutierres, S. (1982). Applications of attribution theory to problems of drug 
abuse: A theoretical and empirical review. Prepared for the National Institute of Drug Abuse, 
grant 1-R101:DA02696, December. 

Baumann, D., Cabianca, W., Cahen, L. (Principle Investigator), Haring, M., Lancy, D., Schultz, 
D.F., & Stout, R. (1982). Athletic Department Academic Program and Tutoring (ADAPT): An 
overview and preliminary evaluation . Prepared for President Page Mulhollan, Arizona State 
University, July. 

Baumann, D., Brown, C., & Paredes, R. (1982). 1980-1981 Annual Report . Prepared for 

Phoenix South Community Mental Health Center. 

Beauvais, C., Baumann, D., Schultz, D.F., & Grigsby, C. (1984). The Austin Homeless: 
Technical Report no. 1. Prepared for the Austin Police Department, Austin, Texas, November. 

Grigsby, C., Baumann, D., Schultz, D.F., & Beauvais, C. (1984). The Austin Homeless: 
Technical Report no. 2. Prepared for the Austin Housing and Community Services Department, 
Austin, Texas, December. 

Schultz, D.F., Grigsby, C., Baumann, D., & Beauvais, C. (1985). The Austin Homeless: 
Technical Report no. 3. Prepared for the Austin/Travis County Community Mental Health 
Center, Austin, Texas, January. 

Baumann, D., Grigsby, C., Beauvais, C., & Schultz, D.F. (1985). The Austin Homeless: 
Technical report no. 4. Prepared for the Austin/Travis County Health Department, Austin, Texas, 
March. 

Baumann, D., Grigsby, C., Beauvais, C., & Schultz, D.F. (1985). The Austin Homeless . Final 
Report. Prepared for the Hogg Foundation for Mental Health, Austin, Texas, May. 

Baumann, D., Grigsby, C., Beauvais, C., Bishop, S., Gregorich, S., Osborne, M., & Nelms, D. 
(1986). Task Force on the Homeless, Volume I: Program Plan and Projected Budget . Prepared 
for the City of Austin, Texas, July. 

Baumann, D., Osborne, M., Beauvais, C., Grigsby, C., Gregorich, S., Bishop, S., & Nelms, D. 
(1986). Task Force on the Homeless, Volume II: Supplement to the Final Report . Prepared for 
the City of Austin, Austin, Texas, August 1. 

Grigsby, C.G., Roberts-Gray, C., & Baumann, D. (1989). Demonstration of Team Case 
Management to Provide Job Training Activities to the Homeless . Prepared for the Department of 
Labor, Employment and Training Administration, Washington, D.C. 

Baumann, D., Grigsby, C.G., & Roberts-Gray, C. (1989). Homelessness in Houston, Harris 
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County, and the Texas Gulf Coast United Way Service Delivery Area . Prepared for the Coalition 
for the Homeless of Houston and Harris County, Houston, Texas. 

Kern, H., Baumann, D., McFadden, T., & Law, J.R. (1993). Caseworker Study Part I: Burnout 
and Turnover . WISDOM Project, Texas Department of Protective and Regulatory Services. 
Prepared for the Department of Health and Human Services, Child Welfare Division, 
Washington, D.C. 

Law, J., Kem, H., Baumann, D., McFadden, T., & Gober, K. (1994). Preliminary Intake Report: 
Part I . WISDOM Project, Texas Department of Protective and Regulatory Services. Prepared for 
the Department of health and Human Services, Child Welfare Division, Washington, D.C. 

Baumann, D., Kern, H.. & Fluke, J. (1997). Worker Improvements to the Decision and Outcome 
Model (WISDOM): The Child Welfare Decision Enhancement Project, Prepared for the 
Department of Health and Human Services, Child Welfare Division, Washington, D.C. 

Baumann, D., Kem, H. & Fluke, J.D. (1997). Foundations of the Decision Making Ecology and 
Overview. In Baumann, D. J.„ Kem, H. & Fluke, J. (Eds.). Worker Improvements to the 
Decision and Outcome Model (WISDOM): The Child Welfare Decision Enhancement Project, 
The Children’s Bureau, Washington, D.C. 

Baumann, D., Kern, H. McFadden, T. & Law, J.R. (1997). Individual and Organizational 
Factors in Burnout and Turnover: A Decision making Ecology Approach. In Baumann, D. J.„ 
Kem, H. & Fluke, J. (Eds.). Worker Improvements to the Decision and Outcome Model 
(WISDOM): The Child Welfare Decision Enhancement Project, The Children’s Bureau, 
Washington, D.C. 

Schultz, D.F., Law, J.R., Baumann, D., Kem, H. & Gober, K.. (1997). Caseworker Studies of 
Individual and Organizational Measures Related to Performance. In Baumann, D. J.,, Kem, H. & 
Fluke, J. (Eds). Worker Improvements to the Decision and Outcome Model (WISDOM): The 
Child Welfare Decision Enhancement Project, The Children’s Bureau, Washington, D.C. 

Baumann, D., Beal, P., Schuman, T., Gober, K., Law, J.R., Kern, H. & Ohmart, R. (1997). 
Critical Incidents Affecting Caseworker Decision Making. In Baumann, D. J.„ Kern, H. & 

Fluke, J. (Eds.). Worker Improvements to the Decision and Outcome Model (WISDOM): The 
Child Welfare Decision Enhancement Project, The Children’s Bureau, Washington, D.C. 

Schwab, J., Baumann, D. & Gober, K. (1997). Patterns of Decision making. In Baumann, D. J.„ 
Kem, H. & Fluke, J. (Eds.). Worker Improvements to the Decision and Outcome Model 
(WISDOM): The Child Welfare Decision Enhancemant Project, The Children’s Bureau, 
Washington, D.C. 

Baumann, D., Esterline, J. A., Zuniga, G., Smith, S., Whiterside, D., Fluke, J., Goertz, B. and 
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Cohen, M. (1997). The Implementation of Risk Assessment. In Baumann, D. J.„ Kern, H. & 
Fluke, J. (Eds). Worker Improvements to the Decision and Outcome Model (WISDOM): The 
Child Welfare Decision Enhancement Project. The Children’s Bureau, Washington, D.C. 

Law, J.R., Kem, H., Schultz, D.F., Gober, K.G., Schwab, J., & Baumann, D. (1997). Screening 
Models for Intake and Investigation. In Baumann, D., Kem, H. & Fluke, J. (Eds.). Worker 
Improvements to the Decision and Outcome Model (WISDOM): The Child Welfare Decision 
Enhancement Project, The Children’s Bureau, Washington, D.C. 

Law, J.R., Baumann, D., Gober, K.G., Schultz, D.F., Ohmart, R., & Kem, H. (1997). Phase III: 
Evaluating the Effectiveness of Risk Assessment. In Baumann, D. J.„ Kern, H. & Fluke, J. 

(Eds.). Worker Improvements to the Decision and Outcome Model (WISDOM): The Child 
Welfare Decision Enhancement Project, The Children’s Bureau, Washington, D.C. 

Gober, K.G., Graham, J.C. Baumann, D.J. & Kern, H. (1998). The Texas Child Fatality Study: 

A Comparison of Fatality and Non-Fatality Cases. Provided to the Texas Department of 
Protective and Regulatory Services. 

Chipley, M.H., Sheets, J.H., Baumann, D.J., Robinson, D. & Graham, J.C. (1999). Flexible 
Response Evaluation . Provided to the Texas Department of Protective and Regulatory Services. 

Graham, J.C. and Baumann, D.J. (1999). Expedited Permanency: Issues and Evaluation . 
Provided to the Texas Department of Protective and Regulatory Services. 

Sheets, J., Caouette, K., & Baumann, D.J.(2000). Alert Pilot; Executive Overview. Provided to 
the Texas Department of Protective and Regulatory Services. 

Sheets, J., Baumann, D.J., Farris, J., Robinson, D., Holoubeck, J., Dodson, D. (2001). High Risk 
Intervention: Stage 1, Implementation Evaluation. Presented to the Texas Department of 
Protective and Regulatory Services. 

Baumann, D.J., Farris, J., Holoubeck, J., Robinson, D., & Sheets (in collaboration with Grigsby, 
W.C., Reid, G., & Graham, J.C.) (2002). Concept Guided Risk Assessment: Reliability and 
Construct Validity. Provided to the Texas Department of Protective and Regulatory Services. 

Baumann, D.J., Law, J.R., Sheets, J., Reid, G., & Graham, J.C. (2002). Evaluating the 
Effectiveness of a Risk Assessment Model. Provided to the Texas Department of Protective and 
Regulatory Services. 

Baumann, D. J., Sheets, J., and Wittenstrom, K., (2006). Family Group Decision-Making: Final 
Evaluation. Provided to the Texas Department of Family and Protective Services . 

Baumann, D. J., Fluke, J., Graham, J. C., Wittenstrom, K., Hedderson, J., Rivaux, S., Dettlaff, 
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A., Rycraft, J., Oritz, M. J., James, J., Kromrei, L., Craig, S., Capouch,D., Sheets, J., Ward, D., 
Breidenbach, R., Hardaway, A., Boudreau, B., & Brown, N. (2010). Disproportionality in child 
protective services: The preliminary results of statewide reform efforts. Austin, TX: Texas 
Department of Family and Protective Services. 

Baumann (2010) Disproportionality in Child Protective Services: 2010 Update (2010). Provided 
to the Texas Department of Family and Protective Services. August 


PROFESSIONAL PRESENTATIONS (approximately 100 presentations after 1995 
available on request) 

Baumann, D., Obitz, F., & Reich, J. (1979). Treatment Induced Reversals in Alcoholics 
Attribution Processes . Paper presented at the Annual Meeting of the Western Psychological 
Association, San Diego, California. 

Baumann, D., Kenrick, D., & Cialdini, R. (1979). The Socialization of Altruism as a Self- 
Reinforcer . Paper presented at the Annual Meeting of the American Psychological Association, 
Toronto, Canada. 

Baumann, D., Sandler, J., & Frost, R. (1981). The Use of Life Events as a Needs Assessment 
Instrument . Paper presented at the Arizona Association of Community Colleges, Casa Grand, 
Arizona. 

Baumann, D., Sandler, I., & Braver, S. (1982). The Use of Life Events as a Method of Needs 
Assessment . Paper presented at the Annual Meeting of the American Psychological Association, 
Washington, D.C. 

Baumann, D., Bettor, S.M., Curtis, S.M., Heller, L.S., Lamb, N.F., Ritter, B.E., Roberts, C.M., 
Wesselo, D.J.(1984). The Day After: An Experimental Investigation of Media Impact . Paper 
presented at the Annual Meeting of the American Psychological Association, Montreal, Canada. 

Baumann, D., (1985). Invited address. The Challenge of Homelessness in Social Science 
Research . Paper presented at the Texas Health and Human Services Coordinating Council, State 
Advisory Committee on the Homeless, Public Forum on the Homeless, Austin, Texas. 

Baumann, D., & Grigsby, C. (1985). Invited address. Applied Social Psychology: Seven Myths . 
Paper presented at the Annual Meeting of the Western Psychological Association, Ft. Worth, 
Texas. 

Baumann, D., (1986). Moral Evaluations and the Law . Paper presented at the Annual Meeting 
of the Society for Experimental Psychologists, Tempe, Arizona. 


Baumann, D., (1988). Who Are the Homeless . Paper presented at a conference entitled 
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"Translating Health Care Research Into Practice." Austin, TX. 

Baumann, D., (1989). Homelessness and Housing . Invited address. Paper presented to the 
Texas Senate, Special Subcommittee on Housing, Austin, Texas. 

Baumann, D. (1991) Enumerating the Homeless . Invited address. Paper presented at the Urban 
Institute and U.S. Bureau of the Census Joint Conference on Methods of Enumerating the 
Homeless. Washington, D.C. 

Baumann, D., & Kennerson, M., (1992). Outcome Measures in Child Welfare. Paper presented 
at the National Conference of Public Administrators in Child Welfare. Santa Fe, New Mexico. 

Baumann, D. (1993). Optimization Models in Human Services . Paper presented at the Workload 
Analysis and Resource Management Workshop. The National Resource Center on Child Abuse 
and Neglect of the National Center on Child Abuse and Neglect. American Humane 
Association, Denver, Colorado. 

Sheets, D., Kem, H., & Baumann, D. (1993). Texas WISDOM: A Three Year Research Project 
to Build Effective CPS Decision Making Instruments . Paper presented at the Seventh National 
Roundtable on Risk Assessment, San Francisco, California. 

Fluke, J., & Baumann, D. (1995). Decision Making in Child Welfare . Paper presented at the 
2nd Annual Family Preservation and Family Support Conference, San Francisco, California. 

GRANTS AND RESEARCH SUPPORT: 

Baumann, D. Behavioral Health Needs Assessment Study in the Center West Catchment 
Area of Maricopa County in the State of Arizona. Contract: 26-235-860-50-3-4-89-0841 between 
Community Behavioral Sciences and State of Arizona. June, 1976. 

Baumann, D., & Hansen, D. Farmworker Energy Project. Funded by Campesinos Unidos, Inc. 
November, 1977. Amount awarded: $21,900. 

Baumann, D., & Hansen, D. Community Services Administration Flood Relief. Program funded 
by Community Services Administration. January, 1978. Amount awarded: $70,000. 

Baumann, D., & Hansen, D. Farmworker Weatherization Project. Funded by National 
Association of Farmworker Organizations. April, 1978. Amount awarded: $14,000. 

Hansen D. & Baumann, D. Farmworker Innovative Residential Skills 
Training. Funded by the Department of Labor, Office of Farmworkers 
Program. April, 1978. Amount awarded: $800,000. 
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Baumann, D. Migrant Head Start Program. Funded by the Department of Health, Education, 
and Welfare. June, 1978. Amount awarded: $137,000. 

Baumann, D., & Hansen, D. DOL/FMHA RA Work Experience Program. Funded by the 
Department of Labor. July, 1978. Amount awarded: $229,700. 

Baumann, D., Eblen, C., & Hansen, D. Statewide, Process Consultation Oriented, Community 
Organization Program. Funded by Champaign for Human Development. July, 1979. Amount 
awarded: $60,000. 

Baumann, D. Life events as a method of needs assessment. Funded by University of Texas 
Research Institute. Summer Research Award. June, 1982. Amount awarded: $4,222. 

Baumann, D.J., Grigsby, C., Beauvais, C., & Schultz, D.F. The Austin Homeless. Funded by the 
Hogg Foundation for Mental Health. June, 1984. Amount awarded: $21,000. 

Baumann, D.J., Grigsby, C., Beauvais, C., & Schultz, D.F. The Development of a Model 
Interagency Program Serving the Austin Homeless. Funded by the City of Austin. November, 
1985. Amount awarded: $9,000. 

Baumann, D.J., Grigsby, C., & Beauvais, C. The Development of a Model Interagency Program 
Serving the Austin Homeless. Funded by the City of Austin. February, 1986. Amount awarded: 
$9,900. 

Baumann, D. Criminalization of the Mentally Ill. Funded by the Policy Research Institute, 
University of Texas. June, 1987. Amount awarded: $3,400. 

Grigsby, C., Baumann, D.J., Roberts-Gray, C. Demonstration of Team Case Management to 
Provide Job Training Activities for the Homeless. Funded by the Department of Labor. October, 
1988. Amount awarded: $245,000. 

Baumann, D.J., Grigsby, C.G., & Roberts-Gray, C. Homelessness in Houston, Harris County. 
Funded by the City of Houston, Harris County, the United Way of the Texas Gulf Coast and 
McKinsey and Company, March, 1989. Amount awarded: $90,000. 

Kem, H., Baumann, D. J., & Fluke, J. The Child Welfare Decision Enhancement Project. 
Funded by the Department of Health and Human Services, Children’s Bureau, October, 1993. 
Amount awarded for three years 390,000. 

Kem, H., & Baumann, D. J., The Screening at Intake Project. Funded through the Child Abuse 
Prevention and Treatment Act, Department of Health and Human Services, Children’s Bureau, 
October, 1994. Amount awarded for five years, approximately $ 100, 000 per year. 
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Baumann, D. J., The Evaluation of Legislative Initiatives Project. Funded through the Child 
Abuse Prevention and Treatment Act, Department of Health and Human Services, Children’s 
Bureau, October, 1999 - October 2011. Amount awarded approximately $ 200, 000 per year. 

Baumann, D. J. and Fluke, J. and Hedderson, J. Disproportionality. Funded by Casey Family 
Programs. January 2008. Amount awarded approximately $ 350,000 

Baumann, D. J. Disproportionality. Funded by the American Humane Association. January 
2008.Amount awarded approximately $ 150,000 


REFERENCES: 

Audrey Deckinga (retired) 
Associate Commissioner 
Texas DFPS 
701 W. 51 st . Street 
Austin, Texas 78714 
(512) 250-9267 

Dr. Marc Lewis 
Associate Professor 
Department of Psychology 
University of Texas 
Austin, Texas 78712 
(512) 471-3393 

Dr. Frank Schultz 
Clinical Psychologist 
P.O. Box 92236 
Lakeland, FL 33804-2236 
(863) 965-7078 


Dr. James Schwab 
Professor 

Department of Social Work 
University of Texas 
Austin, Texas 78712 
(512) 471-9816 

Dr. Homer Kern 
President 
K & K Consulting 
223 Indian Hill. 

China Springs, Texas 76633 
(254) 836-0325 

Dr. John G. Ford 
Professor (retired) 

Dept, of Psychology 
Trinity University 
San Antonio, Texas 
(210) 736-8323 
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Windy M. Hill 
147 Cellus Lane 
Bastrop, TX 78602 
(512) 718-5178 (cell) 

Career Objective: Senior-level Consultative or Management Position 
Summary of Qualifications 

Fifteen years of management experience in creating, developing, implementing, and managing effective 
and fiscally sound federal, state, and local programs. Expertise includes nonprofit capacity building and 
management; community outreach, engagement, and development; project development and 
management; and resource development and grassroots fundraising. 

Professional Experience 

Texas Health and Human Services Commission, Austin, TX 

Program Specialist, Center for the Elimination of Disproportionality and Disparities 
October 1, 2010 - Present 

Perform highly advanced consultative services and technical assistance that involves planning, developing 
and implementing strategic plans; developing goals and objectives to address disproportionality and 
disparities to improve and promote the health status of minority and underserved populations in Texas. 
Responsibilities include analyzing and developing action plans to improve or initiate state and federal 
programs; oversee the completion and monitoring of required reports and recommending changes; 
overseeing studies and analysis of HHS programs; providing consultative services and technical 
assistance to regional partnerships; providing guidance to HHS staff in the development and integration of 
disproportionality and disparities activities and applying culturally competent health services; 
representing HHSC at various meetings, oversight groups and entities related to functions and activities of 
the Center; and coordinating activities with other entities including state and local stakeholders and 
agencies. 

Texas Health and Human Services Commission, Austin, TX 
Director, Office of Community Collaboration 
September 1, 2007 - September 30, 2010 

Managed the delivery of regional community collaboration services for the Health and Human Services 
Commission (HHSC). Specific duties included directing the planning and coordination of regional 
community outreach activities; leading the ongoing transformation of services provided to HHSC 
agencies; developing policies and procedures related to community collaboration programs; representing 
HHSC at various meetings, oversight groups and entities related to these functions and activities; and 
coordinating activities with other entities including state and local stakeholders and agencies. 

Texas Health and Human Services Commission, Austin, TX 
Community Liaison, Office of Communications 
October 1, 2006 - August 31, 2007 

Promoted HHSC coordination and collaboration with community-based organizations to expand access to 
the Medicaid and CHIP programs and services at the local level. Specific duties included conducting 
research and gathering data about state, local, and community Medicaid and CHIP outreach activities and 
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events; coordinating outreach activities with internal and external stakeholders; and conducting local 
public awareness trainings. 

U. S. Department of Health and Human Services, Washington, DC 

Associate Commissioner for the Head Start Bureau, Administration for Children & Families 
January 6, 2002 - May 31, 2005 

Administered the federal Head Start program with a budget of more than $6.7 billion in fiscal year 2004, 
which served more than 900,000 low-income preschool children and their families annually. Specific 
duties included developing, analyzing, and advising on legislative and budgetary proposals; identifying 
areas of research, demonstration, and development; providing leadership and coordination for the 
activities of the program in headquarters and regional offices; representing the Head Start Bureau in inter¬ 
agency activities with other federal and non-federal organizations; and promoting and communicating 
information about program operations and Administration proposals through the media, Congressional 
briefings, and White House Domestic Policy forums. 

Cen-Tex Family Services, Inc., Bastrop, TX 

Executive Director 

January 1, 1994 - January 5, 2002 

Administered the coiporation’s programs, oversaw finances, and supervised and directed the personnel 
for a 501(c)(3) private nonprofit corporation. Specific duties included acting as the coiporation’s general 
manager and fiscal agent; interpreting Board directives and regulatory guidelines and procedures to staff, 
parents, and community stakeholders; creating, planning, implementing, and directing the agency’s 
curricular and extracurricular programs in compliance with federal and state policies and regulations; 
developing and managing programs and projects; managing more than $2 million in grant awards; and 
providing overall direction, coordination, and evaluation of the agency’s 51 full and part-time staff. 

Texas Education Agency, Austin, TX 

Senior Administrative Assistant, Technology Applications Division 
February 1, 1993 - December 31, 1993 

Managed day-to-day administrative activities in the office of an Associate Commissioner. Specific duties 
included coordinating meetings, travel, and conferences; the preparation of presentations, materials, and 
routine correspondence; performing research, and maintaining department-wide time and leave reporting. 

Texas State Library, Austin, TX 
Office Manager, Local Records Division 
October 1, 1978 - December 31, 1993 

Managed the day-to-day operations of the office and worked with local government and state officials in 
the acquisition and preservation of historic local government records. Specific duties included routinely 
communicating technical information about regulations and laws to the public, local, and state officials, 
by telephone and in-person; the physical inventorying, cataloguing, and acquisition of local county 
records; and managing the office’s administrative workflow. 

Education 

Bachelor of Business Administration, Saint Edwards University, Austin, TX 2008 
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IBDO 


Tel: 312-856-9100 
Fax: 312-856-1379 
www.bdo.com 


330 N. Wabash Avenue, Suite 3200 
Chicago, IL 60611 


Independent Auditor’s Report 


To the Board of Directors 
Chapin Hall Center for Children 
Chicago, Illinois 

Report on the Financial Statements 

We have audited the accompanying financial statements of Chapin Hall Center for Children 
(“Chapin Hall”), which comprise the statements of financial position as of June 30, 2014 and 
2013, and the related statements of activities and cash flows for the years then ended, and the 
related notes to the financial statements. 

Management’s Responsibility for the Financial Statements 

Management is responsible for the preparation and fair presentation of these financial 
statements in accordance with accounting principles generally accepted in the United States of 
America; this includes the design, implementation, and maintenance of internal control 
relevant to the preparation and fair presentation of financial statements that are free from 
material misstatement, whether due to fraud or error. 

Auditor’s Responsibility 

Our responsibility is to express an opinion on these financial statements based on our audits. We 
conducted our audits in accordance with auditing standards generally accepted in the United 
States of America. Those standards require that we plan and perform the audit to obtain 
reasonable assurance about whether the financial statements are free from material 
misstatement. 

An audit involves performing procedures to obtain audit evidence about the amounts and 
disclosures in the financial statements. The procedures selected depend on the auditor’s 
judgment, including the assessment of the risks of material misstatement of the financial 
statements, whether due to fraud or error. In making those risk assessments, the auditor 
considers internal control relevant to the entity’s preparation and fair presentation of the 
financial statements in order to design audit procedures that are appropriate in the 
circumstances, but not for the purpose of expressing an opinion on the effectiveness of the 
entity’s internal control. Accordingly, we express no such opinion. An audit also includes 
evaluating the appropriateness of accounting policies used and the reasonableness of significant 
accounting estimates made by management, as well as evaluating the overall presentation of 
the financial statements. 

We believe that the audit evidence we have obtained is sufficient and appropriate to provide a 
basis for our audit opinion. 


BDO USA, LLP, a Delaware limited liability partnership, is the U.S. member of BDO International Limited, a UK company limited by guarantee, and forms part of 
the international BDO network of independent member firms. 


BDO is the brand name for the BDO network and for each of the BDO Member Firms. 
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I BDO 


Opinion 

In our opinion, the financial statements referred to above present fairly, in all material 
respects, the financial position of Chapin Hall as of June 30, 2014 and 2013, and the changes in 
its net assets and its cash flows for the years then ended in accordance with accounting 
principles generally accepted in the United States of America. 

'BOO u<,a,u.P 

Chicago, Illinois 
December 9, 2014 


BDO USA, LLP, a Delaware limited liability partnership, is the U.S. member of BDO International Limited, a UK company limited by guarantee, and forms part of 
the international BDO network of independent member firms. 

BDO is the brand name for the BDO network and for each of the BDO Member Firms. 
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Chapin Hall Center for Children 

Statements of Financial Position 


June 30, 

Assets 

Current Assets 

Cash and cash equivalents 
Amounts held by the University of Chicago 
Operating cash 
Investments at fair value 
Grants, contracts and other receivables 
Prepaid expenses 

Deferred unexecuted contract expenses 
Total Current Assets 

Fixed Assets, net of accumulated depreciation 
Total Assets 

Liabilities and Net Assets 

Accounts payable and accrued liabilities 
Amounts due to the University of Chicago 
Deferred revenue 
Mortgage payable, current portion 

Total Current Liabilities 

Interest rate swap liability 
Mortgage payable, less current portion 

Total Liabilities 

Net Assets 

Unrestricted 

Operating (includes building operations) 
Funds functioning as endowment 
Special funds 

Total unrestricted net assets 
Permanently restricted net assets 
Total Net Assets 
Total Liabilities and Net Assets 


2014 2013 








$ 





See accompanying notes to financial statements. 
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Chapin Hall Center for Children 

Statements of Activities 


Year ended June 30, 2014 

Revenues 

Grants and contracts 
Contributions and reimbursements 
Allocation from investments 
Rental income 

Total revenues 

Expenses 

Research grants and contracts 
General and administrative 
Fundraising 

Total expenses 

Excess of (Expenses)/Revenues 
Before Other Items 

Balance of return on investments 

Change in fair value of interest rate swap 

Increase in Net Assets 

Net Assets, beginning of year 

Net Assets, end of year 


Unrestricted Total Permanently 

Research Building Unrestricted Restricted 


Total 





II 







See accompanying notes to financial statements. 
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Year ended June 30, 2013 


Revenues 

Grants and contracts 
Contributions and reimbursements 
Allocation from investments 
Rental income 

Total revenues 

Expenses 

Research grants and contracts 
General and administrative 
Fundraising 

Total expenses 

Excess of (Expenses)/Revenues 
Before Other Items 

Balance of return on investments 

Change in fair value of interest rate swap 

(Decrease) Increase in Net Assets 

Net Assets, beginning of year 

Net Assets, end of year 


</> </> 


in Hall Center for Children 

Statements of Activities 


Unrestricted Total Permanently 

Research Building Unrestricted Restricted Total 



See accompanying notes to financial statements. 
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Chapin Hall Center for Children 

Statements of Cash Flows 


Year ended June 30, 2014 


Research Building Total 


Cash Flows From Operating Activities 

Increase in net assets 

Adjustments to reconcile change in net assets to 
net cash provided by operating activities 
Depreciation 

Change in fair value of interest rate swap 
Unrealized gain on investments 
Realized gain on sales of investments 
Changes in: 

Grants, contracts and other receivables 
Prepaid expenses 

Deferred unexecuted contract expenses 
Accounts payable and accrued liabilities 
Amounts due to the University of Chicago 
Deferred revenue 

Net cash provided by operating activities 



Cash Flows From Investing Activities 
Fixed asset additions 
Purchases of investments 
Proceeds from sales of investments 



Net cash provided by (used in) investing activities I^BSj 



1 


Cash Flows From Financing Activities 

Principal payments on mortgage payable - | 


1 1 


Net cash used in financing activities - | 


1 1 


Increase in Cash and Cash Equivalents WKBKKl 

Cash and Cash Equivalents, at beginning of year HIHB 

■ 


1 


Cash and Cash Equivalents, at end of year $ 

$ 

$ 


Supplemental Disclosure of Cash Flow Information 
Interest paid $ 


$ 


$ 



Cash and Cash Equivalents at June 30, 2014 Consist of: 

Primary accounts 

Held by the University of Chicago 



See accompanying notes to financial statements. 
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Chapin Hall Center for Children 

Statements of Cash Flows 


Year ended June 30, 2013 


Research Building Total 


Cash Flows From Operating Activities 

(Decrease) increase in net assets 
Adjustments to reconcile change in net assets to 
net cash (used in) provided by operating activities 
Depreciation 

Change in fair value of interest rate swap 
Unrealized gain on investments 
Realized gain on sales of investments 
Changes in: 

Grants, contracts and other receivables 
Prepaid expenses 

Deferred unexecuted contract expenses 
Accounts payable and accrued liabilities 
Amounts due to the University of Chicago 
Deferred revenue 



Net cash (used in) provided by operating activities 


Cash Flows From Investing Activities 
Fixed asset additions 
Purchases of investments 
Proceeds from sales of investments 



Net cash provided by (used in) investing activities 



1 1 


Cash Flows From Financing Activities 

Principal payments on mortgage payable - j 


1 


Net cash used in financing activities - [ 


1 


Increase in Cash and Cash Equivalents 9HH 

Cash and Cash Equivalents, at beginning of year 

1 


1 


Cash and Cash Equivalents, at end of year $ BIHHi 

$ HBi 


Supplemental Disclosure of Cash Flow Information 

Interest paid $ $ $ 


Cash and Cash Equivalents at June 30, 2013 Consist of: 
Primary accounts 
Held by the University of Chicago 



See accompanying notes to financial statements. 
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Chapin Hall Center for Children 

Notes to Financial Statements 


1. Organization 

Chapin Hall is an Illinois not-for-profit corporation that qualifies for exemption from federal 
income taxes under Section 501(c)(3) of the Internal Revenue Code. Chapin Hall conducts its 
operations at the University of Chicago (see Notes 5 and 7). On July 1, 2013, Chapin Hall became 
operationally separate from the University of Chicago (the “University”). Chapin Hall, although 
still formally affiliated with the University of Chicago through its affiliation agreement, became 
responsible for all accounting, employment, and grants and contracts activity. Chapin Hall’s 
endowment continues to be managed by the University. 

Chapin Hall conducts research on, and demonstrates possible solutions to, children’s issues. 
Income from Chapin Hall’s investments, project grants and contracts from governmental and 
private sources, and other revenue fund these projects. Some of Chapin Hall’s research grants and 
contracts from governmental and private sources were arranged through the University for fiscal 
years 2014 and 2013. 

2. Summary of Significant Accounting Policies 

Basis of Accounting 

The financial statements are prepared in conformity with accounting principles applicable to 
nonprofit organizations. 

Display of Assets by Class 

The net assets of Chapin Hall are reported in the following classes: (a) unrestricted net assets 
which are composed of operating funds (which includes building operations), (b) funds functioning 
as endowments and special funds and (c) permanently restricted net assets. 

Management has designated certain funds as “funds functioning as endowment” which include 
donated funds for research and their related income and appreciation. Management has 
maintained these funds as an endowment and intends to continue this practice in the future. 

Management has also designated certain special funds, of which the description and balance of 
funds available are as follows: 

June 30, _2014_2013 


The Strategic Opportunities Funds are used for discretionary projects related 
to the mission of Chapin Hall such as research start-up costs, fund-matching 
costs, organizational and research capacity building expenses and research 
project expansion. $ 

The Center for State Foster Care and Adoption Data (“CSFCAD”) aggregates 
the fees generated from individual state center service contracts, net of 
administrative overhead. The intention of this fund is to use it for costs 
related to this project. 

The Harold Richman Fellowship Fund is a collaborative partnership between 
Chapin Hall and the University of Chicago School of Social Service 
Administration ("SSA") to support young and early career scholars interested 
in conducting applied research related to vulnerable children, families, and 
their communities. 


$ 


$ 


$ 
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Chapin Hall Center for Children 

Notes to Financial Statements 


During fiscal 1998, Chapin Hall received a grant from the MacArthur Foundation in the amount of 
$5,000,000. The purpose of the grant was to contribute to Chapin Hall’s Endowment Fund. The 
principal of the endowment is to be kept intact and the income used for Chapin Hall’s work. The 
grant is reflected as a permanently restricted net asset. See Note 6. 

Financial Instruments and Concentration of Credit Risk 

Chapin Hall maintains its cash in bank deposit accounts, which at times may exceed federally 
insured limits. Chapin Hall has not experienced any losses in such accounts and management does 
not believe Chapin Hall is exposed to any significant credit risk on cash. 

Chapin Hall’s financial instruments are cash and cash equivalents, investments, receivables, 
accounts payable and mortgage payable. The recorded values of cash, accounts receivable and 
accounts payable approximate their fair values based on their short-term nature. The recorded 
value of the mortgage payable approximates fair value, as interest approximates market rates. 

Chapin Hall accounts for the value of its interest rate swap pursuant to Accounting Standards 
Codification (“ASC”) 815, “Accounting for Derivative Instruments and Hedging Activities.” The 
interest rate swap is marked to market based upon quotations from market makers. 

Cash and Cash Equivalents 

Chapin Hall considers all short-term, highly liquid investments to be cash equivalents. Cash and 
cash equivalents include cash, a certificate of deposit, short-term money market funds and 
operating cash amounts on deposit with the University (see Note 7). 

Grants, Contracts and Other Receivables 

Grants, contracts and other receivables consist primarily of amounts due under various contracts 
and grants expected to be collected in the fiscal year ending June 30, 2015. 

Fixed Assets 

Building renovations, furniture and equipment are recorded at cost or, if donated, at estimated 
fair market value at the date of donation. 

Chapin Hall provides for depreciation of fixed assets on a straight-line basis over their estimated 
useful lives - the lesser of estimated useful life or remaining building lease term for renovations, 
10 years for furniture, 3 to 5 years for equipment. 

Amounts Due to the University of Chicago 

Amounts represent funds spent by Chapin Hall during the year on grants or contracts for which the 
University had or had not advanced such funds (See Note 7). 


11 




Chapin Hall Center for Children 

Notes to Financial Statements 


Revenue Recognition 

Contributions are recognized as revenue in the period received or in the period in which an 
unconditional pledge is received. Donor-restricted contributions whose restrictions are met in the 
year of the contribution are reported as unrestricted support. 

Deferred revenue represents operating grant amounts received by the University on behalf of 
Chapin Hall, or directly by Chapin Hall, in excess of the related grant expenditures made. 
Operating grant or contract revenue is recorded for financial reporting purposes as the related 
fully executed reimbursable grant or contract expenditures are incurred. If reimbursable grant or 
contract expenditures are incurred before the grant or contract is fully executed, these expenses 
are recorded as deferred unexecuted contract expenses. 

Expenses 

Chapin Hall has categorized expenses on a functional basis as follows: 

Research Grants and Contracts. This includes research salaries and other direct and indirect 
costs allocable to grant and contract research projects. These expenses are primarily supported 
with grant and contract funds restricted to specific projects. In addition, the total expenses 
include internally funded research. 

General and Administrative. This includes Chapin Hall management, communications and public 
affairs, clerical, accounting staff and other associated overhead expenses. Chapin Hall has 
separately presented the operations of the building it manages on behalf of the University at 1313 
East 60th Street within unrestricted activities. 

Chapin Hall has allocated $509,474 and $521,036 of general and administrative building expenses 
to research operations as rent expense for its 2014 and 2013 cost of occupancy at 1313 East 60th 
Street. 

Fundraising. This includes allocable salaries of management and staff and other direct expenses 
used to solicit unrestricted contributions. 

Income Taxes 

Chapin Hall is exempt from income taxes under Section 501(c)(3) of the Internal Revenue Code 
and applicable state law. Chapin Hall is not subject to the federal excise tax on net investment 
income. 

Uncertainty in Income Taxes 

Chapin Hall follows ASC 740-10-25, “Accounting for Uncertainty in Income Taxes.” Under ASC 740- 
10-25, an organization much recognize the tax benefit associated with tax positions taken for tax 
return purposes when it is more-likely-than-not that the position will be sustained. Chapin Hall 
does not believe that there are any unrecognized tax benefits that should be recorded. No 
interest or penalties are recorded or included in the statements of activities for the years ended 
June 30, 2014 and 2013. Should Chapin Hall need to accrue interest or penalties on uncertain tax 
positions, it would recognize the interest as interest expense and the penalties as other expenses. 
Tax years going back to 2011 remain open to examination by taxing authorities. 
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Fair Value Measurements 

Chapin Hall performs fair value measurements in accordance with ASC 820, “Fair Value 
Measurements and Disclosures,” which defines fair value, establishes a framework for measuring 
fair value in generally accepted accounting principles and expands disclosures about fair value 
measurements. 

ASC 820 defines fair value as the exchange price that would be received for an asset or paid to 
transfer a liability (an exit price) in the most advantageous market for the asset or liability in an 
orderly transaction. Fair value measurement is based on a hierarchy of observable or unobservable 
inputs. The standard describes three levels of inputs that may be used to measure fair value. 

Level 1 - Inputs to the valuation methodology are quoted prices available in active markets for 
identical investments as of the reporting date; 

Level 2 - Inputs to the valuation methodology are other than quoted prices in active markets, 
which are either directly or indirectly observable as of the reporting date, and the fair 
value can be determined through the use of models or other valuation methodologies; 
and 

Level 3 - Inputs to the valuation methodology are unobservable inputs in situations where there 
is little or no market activity of the asset and liability and the reporting entity makes 
estimates and assumptions relating to the pricing of the asset or liability including 
assumptions regarding risk. 

ASC 820 permits a reporting entity to measure the fair value of an investment that does not have a 
readily determinable fair value, based on the net asset value per share (the “NAV”) of the 
investment as a practical expedient, without further adjustment, unless it is probable that the 
investment will be sold at a value significantly different than the NAV. If the practical expedient 
NAV is not as of the reporting entity’s measurement date, then the NAV should be adjusted to 
reflect any significant events that may change the valuation. The investment interest held by 
Chapin Hall in the University Total Return Investment Portfolio managed pool investment fund is 
valued based on Chapin Hall’s NAV in the investment. 

In May 2011, the Financial Accounting Standards Board issued Accounting Standards Update 
(“ASU”) 2011-04, “Amendments to Achieve Common Fair Value Measurement and Disclosure 
Requirements in U.S. GAAP and International Financial Reporting Standards,” to Topic 820 of the 
Codification. ASU 2011-04 changes certain fair value measurement principles and enhances the 
disclosure requirements particularly for Level 3 fair value measurements. Effective July 1, 2012, 
Chapin Hall adopted the disclosure requirements of ASU 2011-04. The adoption did not have an 
impact on Chapin Hall’s financial statements as this guidance relates only to additional 
disclosures. See Note 3. 

Chapin Hall uses an interest rate swap to manage interest rate risk. The valuation of the 
instrument is determined using widely accepted valuation techniques including discounted cash 
flow analysis on the expected cash flows of the interest rate swap. This analysis reflects the 
contractual terms of the interest rate swap, including the period to maturity, and uses observable 
market-based inputs, including the Securities Industry Financial Markets Association (“SIFMA”) 
index. The fair value estimate is classified as Level 2 under the fair value hierarchy. 
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Use of Estimates 

The preparation for financial statements in conformity with accounting principles generally 
accepted in the United States of America requires management to make estimates and 
assumptions that affect the reported amounts of assets and liabilities and disclosure of contingent 
assets and liabilities at the date of the financial statements, as well as the reported amounts of 
revenue and expenses during the reporting period. Actual results could differ from those 
estimates. 

Reclassifica tions 

Certain 2013 amounts have been reclassified to conform to the 2014 presentation. 

Subsequent Events 

The Chapin Hall has evaluated subsequent events through December 9, 2014, the date the 
financial statements were available for issuance. No events have occurred through that date 
which required recognition or disclosure in these financial statements. 

3. Investments 

Chapin Hall invests in the University Total Return Investment Portfolio (“TRIP”) managed pool 
investment fund. The TRIP’S investments in equity securities and debt securities are carried at fair 
value, based on quoted market prices. The TRIP’S investments in alternative investments, such as 
absolute returns, private equities and real assets, are valued at fair value based on valuations and 
other financial information provided by the external investment managers. The valuations of the 
alternative investments involve estimates, appraisals, assumptions and methods which are 
reviewed by the University. Accordingly, the estimated fair values of the alternative investments 
may differ significantly from the values that would have been used had a ready market existed for 
these investments and the differences could be material. The fair value of the TRIP as of June 30, 
2014 and 2013 was $31,268,051 and $28,902,789, respectively. 

The assets of the University TRIP managed pooled investments fund were allocated as follows: 


June 30, 

2014 

2013 

Absolute returns 

26.4% 

24.7% 

Debt securities 

18.8 

19.0 

Public equities 

17.4 

15.8 

Private equities 

15.8 

15.9 

Natural resources 

8.4 

9.8 

Real estate 

7.6 

9.1 

Private debt 

3.0 

3.5 

Cash equivalents 

2.6 

2.2 


100.0% 

100.0% 
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Total return on investments included in the Statements of Activities within changes in unrestricted 
net assets is summarized as follows: 


For the year ended June 30, 

2014 

2013 

Realized gain $ 

Unrealized gain (loss) 

Interest and dividends 

Management fees 

r 

$ 

Total return on investments $ j 


$ 

The total return on investments has been reflected in the Statements of Activities as follows: 

For the year ended June 30, 2014 2013 

Allocation from investments $ 

Balance of return on investments 


n i 

$ I 




The $2,108,466 and $1,955,803 transferred to operations above is the withdrawal from the TRIP 
fund based on the annual endowment calculation in Note 6. This amount is included as 
unrestricted revenue and as an offset to investment income in balance of return on investments 
amount in the Statements of Activities. 

Chapin Hall may request distributions from the TRIP fund once a quarter, subject to the following 
notice requirements: 

Requested Withdrawal Amount _ Notification Period 

Under $10,000,000 90 days 

Between $10,000,000 and $30,000,000 180 days 

Greater than $30,000,000 270 days 

The following tables summarize Chapin Hall’s assets and liabilities accounted for at fair value as 
of June 30, 2014 and 2013, using the fair value hierarchy of ASC 820-10: 


Fair Value Measurement Using 


Quoted Prices 
in Active 
Markets for 

Significant 

Other 

Significant 


Identical 

Observable 

Unobservable 


Assets 

Inputs 

Inputs 

2014 

(Level 1) 

(Level 2) 

(Level 3) 

Total 


Assets 

Cash equivalents - certificate 
of deposit $ 

TRIP managed pool 
investment fund 


-$ 


$ 




Liabilities 
Interest rate swap 


$ 


-$ 



$ 


-$ 



- $ 
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Fair Value Measurement Using 


Quoted Prices 
in Active 

Significant 

Other 

Significant 


Markets for 

Observable 

Unobservable 


Identical Assets 

Inputs 

Inputs 

2013 

(Level 1) 

(Level 2) 

(Level 3) 

Total 


Assets 

Cash equivalents - certificate of deposit $ 
TRIP managed pool investment fund _ 

$ 



Liabilities 
Interest rate swap 


$ 


-$ 




The following table presents a reconciliation of the activity for the Level 3 investments: 


Balance, July 1, 2012 

Net realized and unrealized gains 

Purchases 

Sales 


Balance, June 30, 2013 

Net realized and unrealized gains 

Purchases 

Sales 


Balance, June 30, 2014 



4. Fixed Assets 


Fixed assets consisted of the following: 
June 30, _ 

Building renovations (see Note 5) 

Furniture and equipment _ 

Accumulated depreciation_ 


2014 2013 



5. Mortgage Payable 

In fiscal year 1995, Chapin Hall entered into a mortgage and security agreement with the Illinois 
Educational Facilities Authority (“IEFA”) for the renovation of the University property at 1313 East 
60th Street. Proceeds from the mortgage totaled $5.5 million. 

In May 2003, Chapin Hall refinanced the loan through the issuance of $5,250,000 of bonds secured 
by a mortgage and security agreement with IEFA. During fiscal 2004, the IEFA was consolidated 
into the Illinois Finance Authority. The new bonds are currently enhanced by a letter of credit 
with Fifth Third Bank which extends until May 2017. The balance due on bonds as of June 30, 2014 
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and 2013 was $3,300,000 and $3,550,000, respectively, with interest at a variable rate. The bond 
matures on July 1, 2024. 

The loan is payable over 20 years and future fiscal year principal payments follows. 


2015 

2016 

2017 

2018 
2019 

Thereafter 




Chapin Hall entered into an interest rate swap transaction evidenced by a letter agreement (the 
Agreement) dated May 15, 2003. The Agreement effectively changes Chapin Hall’s interest 
exposure on the outstanding principal balance from a variable rate on the outstanding bonds to a 
quasi-fixed rate, plus basis risk, of 3.95% percent based on an original notional amount of 
$5,250,000, which is reduced over time upon repayment of the bonds. The average effective 
interest rate of the Agreement for the year ended June 30, 2014 and 2013 was 4.07% and 4.06%, 
respectively. The Agreement terminates on July 1, 2024. 


and 


As of June 30, 2014 and 2013, the interest rate swap liability was 
respectively, and is reflected in the Statements of Financial Position and the decrease in fair 
value of $57,878 and $181,351, respectively, for the years ended June 30, 2014 and 2013, is 
reflected in the Statements of Activities. 


6. Endowment Funds 


Chapin Hall’s endowment funds consist of a permanently restricted endowment fund and Board- 
designated endowment funds. As required by accounting principles generally accepted in the 
United States of America, net assets associated with endowment funds, including funds designated 
by the Board to function as endowments, are classified and reported based on the existence or 
absence of donor-imposed restrictions. 

Chapin Hall follows the Uniform Prudent Management of Institutional Funds Act (“UPMIFA”). 
UPMIFA differs from laws previously in place in a few key areas. It eliminates the historic dollar 
value rule with respect to endowment fund spending, it updates the prudence standard for the 
management and investment of charitable funds, and it amends the provisions governing the 
release and modification of restrictions on charitable funds. 

Interpretation of Relevant Law 

The Board of Directors of Chapin Hall has interpreted UPMIFA as requiring the preservation of the 
fair value of the original gift date of the donor restricted endowment funds absent explicit donor 
stipulations to the contrary. As a result of this interpretation, Chapin Hall classified as 
permanently restricted net assets (a) the original value of gifts donated to the permanent 
endowment, (b) the original value of subsequent gifts to the permanent endowment, and (c) 
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accumulations to the permanent endowment made in accordance with the direction of the 
applicable donor gift instrument at the time the accumulation is added to the fund. In accordance 
with UPMIFA, Chapin Hall considers the following factors in making a determination to appropriate 
or accumulate earnings on donor-restricted endowment funds: 

1) The duration and preservation of the endowment fund; 

2) The purpose of Chapin Hall and permanently restricted endowment fund; 

3) General economic conditions; 

4) The possible effect of inflation and deflation; 

5) The expected total return from income and the appreciation of investments; 

6) Other resources of Chapin Hall; and 

7) The investment policies of Chapin Hall. 

Chapin Hall’s endowment net asset composition by type of fund is as follows for the years ended 
June 30, 2014 and 2013. 


Temporarily Permanently 2014 

Unrestricted Restricted Restricted Total 


Permanently restricted 
Board-designated_ 



Temporarily Permanently 2013 

Unrestricted Restricted Restricted Total 


Permanently restricted 
Board-designated_ 


$ 




$ 




The remainder of this page intentionally left blank. 
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The changes in endowment net assets for Chapin Hall were as follows for the years ended June 30, 
2014 and 2013: 


Endowment net assets, June 30, 2012 
Investment return 
Net investment income 
Net appreciation (realized and 
unrealized) 

Total investment return 
Transfer to unrestricted operating 
fund - appropriated 

Endowment net assets, June 30, 2013 
Investment return 
Net investment income 
Net appreciation (realized and 
unrealized) 

Total investment return 
Transfer to unrestricted operating 
fund - appropriated 

Endowment net assets, June 30, 2014 


Temporarily Permanently 

Unrestricted Restricted Restricted Total 


$ 


$ 


$ 


$ 





$ 


$ 




During 2014 and 2013, the permanently restricted fund incurred a total net investment gain of 
$1,091,849 and $557,022, respectively. The gain is reflected as temporarily restricted until its 
appropriation for expenditure in the unrestricted operating fund during 2014 and 2013. 


Funds with Deficiencies 


From time to time, the fair value of assets associated with individual permanently restricted 
endowment funds may fall below the level that the donor or Illinois UPMIFA requires Chapin Hall 
to retain as a fund of perpetual duration. Deficiencies of this nature would be reported in 
unrestricted net assets. There were no such deficiencies as of June 30, 2014 and 2013, 
respectively. 


Endowment Payout 


Chapin Hall has adopted investment and spending policies for endowment assets that attempt to 
provide a predictable stream of funding to programs supported by its endowment while seeking to 
maintain the purchasing power of the endowment assets. Endowment assets include those assets 
of permanently restricted funds that Chapin Hall must hold in perpetuity or for a donor-specified 
period as well as Board-designated funds functioning as endowment. 

Chapin Hall has a policy of a 5 percent distribution each year of its investment portfolio’s average 
fair value, which includes its endowment funds and other unrestricted Board designated funds. 
The average is based on the prior 12 quarters through March 31st of the year preceding the fiscal 
year in which the distribution is planned. In establishing this policy, Chapin Hall considered the 
long-term expected return on its investment portfolio. 
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Chapin Hall’s investment policy provides that the pro-rata share of realized and unrealized gains 
and losses, dividend and interest earned and management fees be allocated to the Board- 
designated endowment fund and the permanently restricted endowment fund in any given year. 
Pay-outs from the endowment for operations are made only out of the unrestricted funds included 
in the investment portfolio. The balance of net assets designated as special funds is not held 
separately in endowment funds. The pay-out for fiscal years 2014 and 2013 was $1,372,189 and 
$1,294,672, respectively. In addition, in fiscal year 2013 and 2014, the Board agreed to a special 
pay-out related to the transition to become operationally separate from the University of Chicago. 
For fiscal years 2014 and 2013, the additional pay-outs were $736,277 and $661,131, respectively, 
based on costs incurred during the year. 

7. Transactions with the University Of Chicago 

Most of Chapin Hall’s research grants and contracts from governmental and private agencies were 
arranged through the University for fiscal years 2014 and 2013. The terms of certain research 
grants and contracts allow the University to charge indirect overhead to the funding agencies. 

Chapin Hall has certain funds on deposit with the University. These funds consist of the following 
three elements. 

a. Operating cash: Investment income allocated to Chapin Hall by the Board of Directors and 
other unrestricted income is deposited with the University and is drawn upon based on 
operating cash needs. Such deposits are noninterest-bearing. 

b. Amounts held by (due to) the University of Chicago: These amounts represent proceeds from 
grants held by the University on Chapin Hall’s behalf and are released as grant-related 
expenditures are incurred or represent funds spent by Chapin Hall on grants or contracts for 
which the University advanced funds. 

c. Investments: Investments in a pooled investment fund are held and managed by the University 
(Note 3). 

Chapin Hall occupies and has renovated a building owned by the University (Notes 4 and 5). The 
lease term is for a period of 30 years expiring on December 31, 2024 with annual base rental 
payments of $100 to the University. The lease agreement stipulates that Chapin Hall is responsible 
for all costs of operating the building. Additionally, Chapin Hall participates in the funding of 
capital improvements. During 2014 and 2013, respectively, the University provided $350,000 to 
reimburse Chapin Hall for certain costs of operating the building. Chapin Hall rents out space to 
various University departments under various short-term lease agreements. In 2014 and 2013, 
rental income from University departments totaled $383,154 and $350,266, respectively. 
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